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RATIONALE 

The College Student Experiences questionnaire, first published 
in 1979, and slightly revised in a t,econd edition published in 
1983, is a unique instrument for assessing the quality of under- 
graduate student education and examining the sources. of student 
progress toward the attainment of important goals of college 
education. The intellectual origins of this instrument come from 
a variety of views and concepts about the nature of higher education, 
about accountability, about student learning and development, and 
about the need for new measures in the evaluation of higher education 
programs. 

The Concept of Quality 

Education is both a process and a product. When we evaluate 

educatio nal prnr|raing, hpwpx/er, we hav^ Ivpi-callv viewed education 

as a product — knowledge acquired, skills improved, attitudes and. 
values modified, personal traits developed. We have typically 
thought of educational processes in terms of what they contribute 
to the product--with some processes or methods judged as better 
than others because they produce more learning, higher test scores, 
or whatever the intended outcome may be. We readily agree that 
[ some products are inherently better than others — knowledge is 
'better than ignorance, tolerance is better the^h bigotry, however 
j they may be attained. Is it not also true that some processes are 
inherently better than others, regardless of whether they produce 
more learning? The process or experience of trying to see how 
things fit together, as in making an outline, is a better educational 
experience than the process of memorizing dates in a history book. 
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The intellectual level of the former is higher than the intellectual 
level of the latter, and that is 50 whether or not it leads to a 
higher score on some achievement test. The experience of seeing 
the Grand Canyon in Arizona is qualitatively different from the 
experience of seeing picWes of it in the National Geographic. 
The value of the experience is inherent in the experience itself. 
Whether it leads to ^ higher score on a geology test is irrelevant. 
So, in thinking aboutVhpw we ^evaluate educational programs it 

seemed "to me that the qfiaTPity of the educational experiencf; or 

process should somehow be taken into account. We need ways to 
measure the quality of the process as well as the quality of the 
product. 

Another line of thinking stems from the fol lowi ng basic fact: 
all learning and development require an investment of time and 
effort by the student. Time is a frequency dimension. . Effort is 
a quality dimension in the sense that some educational processes 
require more effort than others. It's fairly easy to look up a 
given reference in tho library. It's more difficult, takes more 
effort, to develop a set of references for a. report. It ooesn't 
take much effort to look up a word in the dictionary. It takes 
more effort, more initiative, to ask other people to read something 
you have written to see if it is clear to them. In both of these 
examples, the activity requiring the greater effort is also poten- 
tially more educative. It takes more effort to get to the Grand 
Canyon than it takes to get to the library! Quality of experience 
and quality of effort are similar concepts, connected with one . 
another in that the likelihood of having high quality experience 
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depends on investing high quality effort. By measuring "effort" 
we may have the key to judging the quality of the educational 
process. 

Still another line of argument influenced my feeling that 
quality of effort was an important and timely concept. Much of 
the current rhetoric about institutional accountability and con- 
sumerism in higher education is onesided. If students don't 
graduate, the institution. is accountable. If students don't 
learn, the teacher is accountable. If th^^^graduates don't get 
good jobs, the institution is to blame. 

There are many questionable assumptions in this common rhetoric. 
It assumes that leaving college before getting a degree is a sign 
of failure when in many cases it may be a prudent and well-informed 
decision. It assumes that professors produce learning. It assumes 
that the college, not the national econoiny, controls the job 
market. It assumes that if students don't benefit from going to 
college, it's the college's fault. It assumes that the student is 
buying a product when actually the student, at a later point in 
time, is the product. 

Colleges are, of course, accountable for a lot of things. 
They are accountable for the resources and facilities, the programs 
and procedures, the stimuli and standards they provide for student 
learning and development. But surely the students are also account- 
able for the amount, scope, and quality of effort they invest in 
their own learning and development, and specifically in using the 
facilities and opportunities that are available in the college 
setting. Accountability for achievement and related student 
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outcomes must consider both what the institution offers and what 
the students do with those offerings. 

Students know that they are accountable. Asked their opinion 
^bout the following statement ("If students expect to benefit from 
what this college or university has to offer, they have to take 
the initiative") about 95 percent of undergraduates from all over 
the country said they agreed with that statement. They know that 
what they get out of college will depend, to a considerable degree, 
on what they put -into it. 
Determining the Content 

Granted the importance of "quality of effort," one next faces 
the question "Effort at what?" There are a great riany experiences 
in different aspects of college life that contribute to student 
learning and devel^ment. How does one decide what aspects of 
college life to look at? And how does one decide on the underlying 
quality dimension in each of these aspects? There should be some 
theoretical or conceptual backing for the way one answers those 
questions. 

The college experience consists of the events that occur in 
the college environment. Many of these experiences stem from 
events and conditions which the college makes possible and which 
at least in some respect are intended to facilitate student learning 
and development . The most salient of these events and experiences 
ar€ clustered around a number of fairly common facilities 
classrooms, libraries, laboratories, residence units, student 
unions, chapels, athletic spaces, studios, galleries, theatres, 
auditoriums and others. Each facility has a particular purpose 
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and there are characteristic activities that occur in them. Other 
events and experiences, not associated wi th any particular physical 
setting, are also of major importance in college life. These 
would include contacts with faculty members, involvement In student 
clubs and organizations, student friendships, informal conversations, 
and- a great variety of relationships. We decided that the topics 
about which we would construct quality of effort measures should 
include major facilities and major types of opportunities that 
commonly exist in~the college setting. The total number of topics 
should provide reasonable coverage of Important kinds of events 
and experiences. 

Since the focus of the measures would be on student learning 
and development, the selection of content within each measure 
would be guided by basic concepts of learning and developri! t. 
With respect to classroom learning we had already determined that 
the quality dimension would be one of increasingly higher cognitive 
levels. For other topics, concepts about personality development 
woula be more relevant. Personality develops as it encounters new 
experiences v/hich require new modes of response. One does not 
grow without having something to grow on. There must, in other 
words, . *^ some contact, some encounter, some set of events and 
experiences which theoretically reflect increasing levels of 
involvement, challenge, and effort. For still other topics, 
concepts of learning or personality development might not provide 
an appropriate underlying structure. For example, in some aspects 
of the college experience, the concern would be in the way students ) 
usfed a particular physical facility. Here the underlying quality 
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dimension might be simply an educational one which holds that 
quality of effort is suggested by the extent to which students use 
the potential inherent in the facility. In the very broadest 
sense » and probably with respect to all of the quality of effort 
measures, one could say that the underlying^ quality dimension or 
concept wac that of capitalizing on the potential for learning and 
development inherent in the nature of the particular category of 
experience. 

We made four other decisions about the content. We decided 
that we would not consider facilities or experiences off-campus. 
Our focus wotjld be on facilities which the college provides , and 
on experiences or opportunities that exist within the college 
environment . Secondly, we would limit our scope to major facili- 
ties found on all or most caippuses and that all or most students 
would or could readily use. Consequently, we would exclude chapels, 
art studios, shop facilities, etc. Third, in our consideratton of 
student activities we would include only activities that are 
presumed to be desirable in fostering learning and personal develop- 
ment in directions intended by the college. Thus, we would exclude 
interesting but inappropriate activities such as slept in class, 
cheated on a test, drank beer, smoked marijuana, etc., regardless 
of how much effort students may devote to such matters. And 
fourth, we would not be concerned with attitudes, feelings, traumas, 
identity crises, frustrations, or similar matters of clinical 
interest. We would focus, as far as possible, on activities and 
objectively observable behavior. 
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In short, the content of the quality of effort measures would 
be "up front," and would focus, on how students use the major 
resources and opportunities for learning and personal growth that 
are provided by the college for that purpose. 

The topics, or aspects of college, Included 1^^ the subse- y 
quently developed measures Include the following facilities: / 
classroom (courses), library, science facilities, cultural facili- 
ties (art, music, theater) student union, athletic and recreational 
facilities, and residence facilities (dormitory or fraternity/ 
sorority); and the following other opportunities for personal and 
Interpersonal experiences: experience with faculty, clubs and 
organizations, expenences in writing, personal experiences (related 
to self-understanding),, student acquaintances, topics of conversation, 
and Information in conversations. 
T he Method of Measurement 

We have already indicated that quality, of effort would be 
judged by the activities students engaged in, that some activities 
would require more effort than others, and that some activities 
would have a higher quality (greater potential for influencing 
learning or personal growth, or be Inherently "better") than 
others. 

The format for translating these ideas into a measure is a 
list of activities, ranging from ones that are' easy to do to ones 
that require more effort, with students asked to report about how 
often they have engaged in each of the activities during the 
current school year by checking "never," "occasionally," "often," 
or "very often". By giving one point for "never," two points for 
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"occasionally," three points for "often," and four points for 
"very often," one can get a score for the category. Since a high 
score can only be attained by engaging with some frequency in the 
-higher quali'ty activiltes, the score reflects "quality of effort." 
Most of the check- lists devised for each topic consisted of ten 
activities, so a student's score could range from 10 to 40, with 
10 meaning that the student had never engaged in any of the activi- 
ties^ and 40 meaning that the student had engaged "very often" in 
all of the activities. 

This style of measurement can be called a scale . The defini- 
tion of a scale as used here is as follows: 1) the activities 
within it are a coherent universe of content—that is, if the 
scale purports to measure quality of effort in using the library, 
then all items or activities would refer to using the library, 
2) the activities, in so far as possible, would reflect a uni- 
dimensional hierarchy, meaning that they are interdependent, in 
the sense that engagement in the higher quality or most difficult 
activities subsumes engagement in the lower quality or easier 
activities. When these conditions are met, the result is a very 
reliable score which also has a very explicit meaning. In this 
way, it is possible to have reliable measures with a relatively 
small number of items. This type of measurement was originally 
devised by Louis Guttman in the 1940s.. I have previously used the 
model in constructing scales to measure the involvement of college 
graduates in various activities--political , civic, cultural, etc. 
It seemed a natural and suitable solution for a way to measure the 
quality of effort students were investing in using the resources 
and opportunities for their education in college. 
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The topics, previously noted, are listed below with an indica- 
tion of the underlying quality dimension for each topic: 

Classroom (course learning scale) (10 actiyities) 

From: relatively simple cognitive activities--such as taKing 
notes, underlining, etc. 

To: higher level cognitive activities—such as efforts to 

explain iand organize 

Library (10 activities) 

From: routine, moderately exploratory use— such as using the 
card catalogue 

To: increased amount of independent exploration and focused 

activity— as in browsing in the stacks, developing a. 
bibliography 

Facilities related to the Arts (Art, Music, Theater scale) (12 
activities) 

From: talking about and attending 

To: efforts toward greater understanding (seeking the 

views of experts and critics) and personal involvement 

Facilities related to Science/Technology (principles, procedures, and 
computers) (12 activities) 

From: memorizing, watching, reading 

To: efforts to explain, experiment, and develop skills 
Student Union (10 activities) 

From: casual and informal use-- had snacks, met friends, etc. 
To: programmatic use— attended events, held meetings, etc. 

• ■ 

Athletic and Recreation Facilities (10 activities) 

From: generally informal use— exercise, games 

To: greater efforts toward improvement and skilled per- 

formance 

Dormitory or Fraternity/Sorority (10 activities) 
From: general socializing 

BEST COPV AVAILABLE 

. . 12 ■ 



To* more personal exchanges—helping, sharing, studying 

together, working on projects 

Experiences with Faculty (10 activities) 

From: routine and causal 

To: more serious contacts—such as discussing careers, 

inviting criticisms, seeking counsel 

Clubs and Organizations (10 activities) 

From: awareness of events and organizations 

To: attending events, discussing programs, working in 

organizations 

Experiences in Writing (10 activities) 

From: general concern with words, grammar, revisions 

To: seeking criticism from others, greater concern with' 

clarity and style 

Personal Experiences (10 activities) 

From: general curiosity about understanding one' s own 
behavior, and others—talked with friends, etc. 

To: more focused and expertly informed sources of self- 

understanding— as in reading, taking a test, talking 
with a counselor 

Student Acquaintances (10 activities) 

From: making friends with different kinds of peopje-- breadth. 

To: serious conversations with people who differ from 

you— depth 

Topics of Conversation (12 items) 

From: personal and interpersonal topics of immediate experi- 
ence-- jobs, movies, social events 

To: intellectual and cultural topics concerning values and 

social issues 

Information in Conversations (6 activities) 

From: conversations in which information about the topic is 
relatively casual and Infrequently Introduced . 
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To- conversations that typically have expertise, 

knowledge, and persuasiveness brought to bear on the 
topic - / 

These are the topics in, the revised second edition 1983. 
With one exception these are also the topics initially included in 
the 1979 edition. In that earlier edition there was a scale 
labeled Science Laboratory, but it was subsequently replaced by 
the' scale labeled Science/Technology. Many students had been 
unable to answer the items tn the Science Laboratory scale because 
they had not had a "laboratory" course. Also In the Conversation 
Topics scale, the 1983 edition has two additional subjects. The 
overall content of the instrument is nearly the same as the 1979 
edition, and most items are identical. 

In summary, we devised 14 scales (lists of activities which 
reflect increasing levels of effort and potential value), seven 
'* addressed to students' use of major, campus facilities and seven 
addressed to other opportunities for experience in the college en- 
vironment. The responses to these scales can be scored, producing 
a measure of the quality of effort invested in each of the topics 
or aspects of college life. While the measures are not perfect 
scales in the literal definition of a scale by Guttman, and need 
not be so. the idea of a scale influenced the selection of activi- 
ties, and the result, as the sect^ion on RELIABILITY will report, 
is a series of measures that are "similar to" Guttman's concept of 
a scale. 

Viewed in another way. the 142 activities in the 14 quality 
of effort scales provide a systematic inventory of the campus 
experiences of undergraduates. These activities comprise half of 
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the content of the College Student Experiences questionnaire--3'8 
pages out of 7 pages of questionnaire items. 
The Larger Scope of Inquiry ^ 

The total content of the questionnaire is designed to provide 
additional information about the significance of quality of effort 
in the evaluation of higher education. How, for example, is it 
related to the characteristics of students, to their particular 
• status in college, Xo characteristics of the college environment, 
to student satisfaction and progress' toward important goals of 
undergraduate education? 

The first part, of the complete questionnaire consists bf a 
series of Items under the heading. "Background Information." There 
are two types of Information in this section— first, iriformation 
about the individual: namely, age, sex, marital status, racial or 
ethnic identification, educational level of the parents and, in 
the 1983 edition, citizenship and foreign student, identification; 
and second, information about the studfent's status in college: 
year in school , where one lives (dormitory, etc.), whether one is 
a transfer student, grades, ma joV field, expectations about con- 
tinuing for an advanced degree, full-time or part-time attendance, 
amount of time spent studying, amount of time if any spent working 
on a job, and proportion of college expenses paid by parents. or 
family. These items enable one to determine the relationsjiij) 
between quality of effort and Important personal characteristics 
and various conditions in college (major field, residence, grades, 
etc. ). 
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-There are no questions about high" school activities, personality, 
or college aptitude scopes. The main focus of the questionnaire 
is on what student's do in college , and on wh^t conditions in college 

\ ' 

u 

influence what they do and what they achieve. It is in this place 
and time period that college faculty and administrators operate. 
It is possible, although not very probable, that quality of effort 
is determined by the chromosomes; but how does one ask about that 
in a questionnaire! As to personality, a recent study found no 
relationship between quality of effort and any of the personality 
traits measured by'the Meyers-Briggs Type Indicator (the Jungian 
typology of extroversion-introversion; thinking-feeling; judging- 
perceiving; sensing-intuiting). In any case, the College Student 
Experiences questionnaire is not a clinical instrument. There is 
also no question about student's SAT scores, because some students 
don't know the answer, some sti^dents regard the question as an 
invasion of privacy, and it's unnecessary because three questions 
in the background section provide information that correlates well 
with SAT scores— namely, college grades, parents' education, and 
educational plans to continue beyond college for an advanced 
degree. 

Following these "Background Information" items, and following 
the fourteen quality of effort scales, there are two or three 
items of special interest. In the ^979 edition,. one of these 
items was designed to see what students really meant when they had 
answered the quality of effort items by checking "never," occasion- 
ally," "often," and "very often," Seven items from the quality of 
effort scales, representing quite different content, were repeated, 
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with students now asked to indicate the number of times they had 
engaged in that activity; the options now heing "never," "oncf or 
twice during the year," "about three to six times during the 

_year," "^ihout once or twice a month," "about once a week," "n.^re 

than once a week." These questions obviously were meant to clarify 
the meaning of the response categaries. This content was introduced 
because of my personal interest in the psychology of comparative 
judgments, and because the information would have practical bearing 
on inter-institution comjsarisons if-^ "often" at Amherst doesn't 
mean the same as "often" at UCLA. The research on this topic has 
been reported in a journal article by Pace and Friedlander, showing \. 
that the meaning of the response categories was mainly related to 
the topic or content, and slightly related to the school. Having 
satisfied the basic research purpose of asking the question, the 
revised second edition of 1983 did not include that item. Instead, 
a new 'and very timely set of questions was introduced about the 
extent of reading and writing students had done during the current 
school year. 

Also, in both the 1979 and 1983 editions, there were two 
questions that in combination produce an index of students' satis- 
faction with college: How well do you like college? If you could 
start over again would you go to the same college you are now 
attending? Finally, the third question in this brief section was 
the one quoted previously which had shown that students agree (95% 
or so) that "if students expect to benefit from what this college 
' or university has to offer, they have to take the initiative." 
Their response to this question validates. the importance of measuring 
"quality of effort." i 
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The next major section of the questionnaire focuses on char- 
acteristics of the college environment. A great deal of research 



college environment in facilitating students learning and develop- 
ment. Much of this documentation was made possit?le by the inven- 
tion of new measures to characterize the environment— especially 
the College Characteristics Index by Pace and Stern, 1958, and the 
College and University Environment icales (CUES) by Pace, 1963, 
and 1969. These fneasures sought to characterize the "psychological 
climate" of the campus. There have been, and still are, studies 
concluding that the college environment has little impact on 
student achievement, but those studies have usually considered the 
physical and economic environment (size, library holdings, physical 
plant, endowment, tuition, faculty-student ratio, etc.), not the 
psychological environment. Learning and personal development are 
psychological phenomena, not physical or economic phenomena. 

My own studies of -college environments, based on results from 
the College and University Environment Scales, had identified five 
major dimensions along which environments differed, dime- > ions 
labeled Scholarship, Awareness, Community, Propriety, and Practicality. 
Other researchers, using other measures, have always identified 
dimensions very similar to these, sometimes using different labels, 
sometimes describing environments in greater detail, but with 
results that basically confirm the importance of several common 
dimensions. 

One of the most prolific and careful researchers in the study 
of environments has been Rudolph Moos at Stanford. Whether studying 



over the past twenty years has documented the importance of the 





the envirpnments of college residence halls, high school classrooms, 
hospital wards, military units, work environments, etc. , he found 
that there were always three types of elements: personal develop- 
ment or goal elements (aspects that support that main purpose of . 
the environment— such as morale in the military, learning in the 
classroom, etc.); interpersonal relations (especially the extent 
to which people in the environment ate supportive of one another); 
and organizational elements (flexible, adaptive, vs. rigid, rule- 
bound, etc.). 

Nearly all of the creative research about environments over 
the past 20 years has been based on "collective perception" as the 
source of knowledge about the "psychological climate." This 
remains, in my opinion, the most valid way to characterize an 
environment psychologically. For the College Student Experiences 
questionnaire, however, I felt that the collective perception 
method would require too many items to reach satisfactory reli- 
-attlity— Try~compVomise was to ask students to rate the college 
environment, based on their experience in it, along a set of 

characteristics that represented .my own past research and the 
extensive research of Rudolph. Moos. 

In the 1979 edition there were eight rating scales. The 
first four ask students to indicate how much emphasis they feel is 
given at the college to certain aspects of student development. 
Those aspects are emphasis on the development of academic, scholarly, 
and intellectual qualities; emphasis on the development of esthetic, 
expressive, and creative qtjalities; emphasis on being critical, 
evaluative, and analytical; and emphasis on the development of 
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vocational <ind occupatioPui competence. In the 1983 edition a 
fifth rating was introduced regarding emphasis on the personal 
relevance and^actjcal value. of the courses. For each of these 
emphases there is a seven-point rating scale, ranging from strong 
emphasis at one end to weak emphasis at the other. Then there are 
three rating scales that refer to personal relationships within 
the college environment— relationships with other students, rela- 
tionships with faculty members, and relationships with administra- 
tive offices and officials. Here again, on a seven-point scale, 
the student is asked to characterize the nature of those relation- 
ships at his or her college. The relationships with students 
range at one end from friendly and supportive, to uninvolved and 
alienated at the other end. The relationships with faculty members 
range from approachable, helpful , understanding, and encouraging 
at one end of the scale to remote, discouraging, unsympath..tic at 
the other end. The relationships with administrative offices and 
officials range from approachable, helpful, open-minded at one end 
of the scale to rigid, resistant, bound by regulations at the 
opposite end. In the 1979 edition, but not in the 1983 edition, 
there was a scale referring to the general style of operation as 
an organization. One end of that rating scale was described as 
flexible, open, adaptive; the other end of the scale was described 
as remote, difficult, impersonal. These rating scalei, are intended 
to capture in a brief fashion important qualities of the college 
environment: the extent to which it emphasizes certain objectives 
or goals, the nature and quality of personal relationships within 
the environment and particularly the supportiveness of those 
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relationships, and the general flexibility of the organization and 
administration as students perceive it. 

The final section of the questionnaire is called "Estimate of 
Gains." This consists, in the 1979 edition, of eighteen statements 
of fairly typical and important objectives, such as vocational 
training, a broad general education, writingclearly and effectively, 
ability to think analytically and logically, and so forth. For 
each of these eighteen statements, the student is asked: To what 
extent do you feel you have gained or made progress (in college up 
to now?) The student could answer by indicating "very little," 
"some," "quite a bit," or "very much." These self-reportf d gains 
can be regarded as an indication of the extent to w( jn .'■"jants 
believe they are achieving important objectives of college education; 
and one can then determine the extent to which high quality effort 
contributes to high attainment or progress toward related goals. 

In the 1983 edition there are 21 goal 'statements--two-added -ones 

and one edited one. 
Summary of Rationale 

1. Education is both a process and a product. 

2. All learning and development requires an investment of time and 
effort by the students. 

3. Students are accountable for what they put into using the facili- 
ties and opportunities provided by the college for their learning 
and development. 

4. The amount, scope, and quality of studentc' effort is a key to 
identifying the quality of the educational process. 

5. It is this "student activities" element of education as a process 
that has not been well represented in previous evaluations of 
higher education. 
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6. 
7. 

8. 

9. 
10. 

11. 



The topics, or aspects of college life, represented In the 
College Student Experiences questionnaire consist of major 
facilities and major opportunities found on nearly a11 to11ege 
campuses, - - - v 



The Inventory of student activities Is limited to fadllr^s 
provided by the college and opportunities that exist withtn the 
college environment-- thus excluding off-campus activities, 
activities not Intended by the college, and aspects of experi- 
ence that are "primarily Internal or clinical rather than 
external and objectively observable. 

Within each topic or aspect of college experience, the activi- 
ties are Ihtended to form a scale ranging from activities 
requiring relatively little effort to ones requiring much more 
effort and Initiative. 

The form of response Is for students to Indicate the frequency 
of their participation In each of the activities during the 
current school year. 

The complete. College Student Experiences questionnaire also 
Includes Information about student characteristics, about their 
particular position or status In college, their satisfaction 
with college, ratings of characteristics of the college environ- 
ment, and estimates of gain or progress toward Important ob- 
jectives of higher education. 

The fourteen quality of effort scales can be viewed as a battery 
of tests, a set of reliable measures of student activities 
1 ntended^o -prornot*^ TearrrfTig and pcrsbnaT groWtli. Fhe'Tel 1 a- 
billty and other properties of these measures are the subject of 
the next section In this report. 
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RELIABILITY 

Since we have repeatedly referred to all the quality of ^ 
effort activities in the questionnaire as. forming a set of measures 
or scales, a battery of tests rather than some unstructured assort- 
ment of student behavior, we are obliged to examine and report 
their properties as tests so one may judge whether they are good 
tests. The quality of effort measures are somewhat analogous to 
achievement tests. Each measure or test deals with a specific 
topic or aspect of college life; students' responses to the items 
produce a distribution of scores ranging from high to low; the 
reliability of these scores can be estimated by statistical methods; 
and these and other measurement properties, following classical 
test construction theory, are reported 1n this section. What we 
will consider under the heading of reliability is, however, broader 
than the usual textbook definition of test reliability. In a 
larger sense, our concern is with confidence . Are the results on 
the measures congruent with what one might expect from prior 
research and theory? How much credibility can one reasonably 
attribute to students' responses to the questionnaire items? 
Basic Psychometric Data ' 

Before the College Student Experiences questionnaire was ini- 
tially available in 1979, preliminary versions of the various 
measures had been pre-tested and subjected to careful psychometric 
analysis and several revisions had been made to improve them. In 
the spring of 1979 the questionnaire was given to samples of 
students at 13 colleges and universities and new psychometvic 




studies of the finished or final scales were made. The 13 institu- 
tions included two community colleges. Since the tests were 
intended for undergraduates in four-y^ar baccalaureate degree- 
granting institutions, the responses from the community colleges 
and from students who had classified themselves as fifth-year or 
graduate students were eliminated, leaving somewhat over 3,000 
cases to be ainalyzed— from 3 doctoral granting universities, 3 
comprehensive universities, and 5 liberal arts colleges. 

A good test is discriminating, reliable, and valid. Table I 
provides evidence related to discrimination and reliability. The 
items comprising a test should have a positive relation to one 
another, indicating chat they "go together." The item intercor- 
reiations in the table are all positive, albeit some are a bit too 
low and others are a bit too high. On balance. Indicated by the 
median of the intercorrelations of items in each scale, the rela- 
tionships are close to the ideal one seeks in a group of test 
items. The next group of numbers in the table show that, within 
each scale, every item makes a significant and positive contribu- 
tion to the scale score. The scale scores, it will be recalled, 
are obtained by giving 4 points to each activity marked "very 
often," 3 points for "often," 2 for "occasionally," and 1 for 
"never," so, on a ten-item scale the scores could range from 10 to 
40, The means and standard deviations of scores on each scale, 
shown in the table, indicate that there are indeed, individual 
differences in students quality of effort. Since the possible 
range of scores on the 10 item scales is 30 points (10 to 40), and 
since the obtained standard deviations are typically between 5 and 
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7 points, and since, in a normal distribution, two sigmas from the 
mean would encompass 95% of all the scores, the fact that the 
obtained scores range from two-thirds to nine-tenths of this 
theoretical territory indicates a very good spread of individual 
differences. The reliability estimates shown in the tables are 
"Coefficient Alpha," an index that is based partly on the variance 
of scores. These reliability coefficients, unusually high for 
_.^y^!l_short^ tests ».J_ndicatfi. .that the^^-of^es-^H^-dfscnirrinat-rti^-and 
dependable. 

the other two headings in Table 1, showing the results of 
factor analyses and Guttman scale analysis, are concerned with 
test content. A factor analysis was made of the items in each 
scale. If the it,em content is totally coherent and interrelated, 
then all the variance in the test scores would be accounted for by 
a single, totally dominant factor. In two of the quality of 
effort scales this is true; in eight of the other scales, more 
than 80% of the variance is accounted for by one dominant factor; 
and in three scales the percent accounted for is 70% or better. 
Factor analysis and reproducibility data are not shown for the 
art, music, theatre scale because th6 activities refer separately 
to the three topics, and no single hierarchial element was sought. 
On the whole, one can regard the content coherence of the activi- 
ties in each scale as very high. The coefficient of reproducibility 
indicates how well the scale scores meet one definition of a 
"scale." In his early work, Guttman suggested that a coefficient 
of 85% would be d good criterion to use. It means that> knowing 
individual's scores on the test, one could reproduce 85% of all 
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TABLE 1 

QUALITY OF EFFORT SCALES BASIC PSIYCHOMETRIC DATA 



03 



Scales 

1979 Edition 



Number Item 
of Intercorrelations 
Items Raijige Median 



Item- scale 
Correlations 
Range Median 



Distribution 
of Scores 



Mean 



Sigma 



Reli- 
ability 
Estimate 



Percent 
Variance 
First 
Factor 



*Note: Decimal points have been omitted from all correlations. 



Coefficient 
of Repro- 
ducibility 



Library 


10 


0.4 


to 


61 


26 


16 


to 


62 


51 


19.2 


4.9 


79 


88 


83 . 


Faculty 


10 


18 


to 


63 


44 


36 


to 


71 


64 


19.1 


5.4 


82 


100 


91 


Course Learning 


10 


. 08 


to 


59 


27 


36 


to 


58 


47 . 


29.7 


5.0 


79 


70 


84 


Arts, Music, Theater 


12 


06 


to 


66 


26 


32 


to 


66 


46 


20.1 


6.2 


83 




mm 


Student Union 


10 


12 


to 


77 


30 


24 


to 


71 


51 


20 9 


5 9 






OQ 


Athletic Facilities 


10 


25 


to 


69 


46 


48 


to 


73 


66 


, 17.5 


6.9 


89 


88 


88 


Clubs 


10 


20 


to 


69 


46 


41 


to 


75 


67 


19.2 


6.8 


90 


89 


88 


wr 1 1.1 ny 


1 n 
lU 


AD 

Uo 


to 


in 
f\J 


33 


19 


to 


61 


57 


24. 3 


5.9 


84 


82 


86 


Personal Experiences 


10 


08 


to 


63 


26 


24 


tc 


60 


51 


21.8 


5.8 


8»* 


82 


81 


Student Acquaintances 


10 


.13 


to 


67 


36 


44 


to 


66 


58 


23.9 


6.3 


87 


74 


81 . 


Dormitory F/S 


10 


22 


to 


65 


44 


50 


to 


70 


64 


25.1 


6.8 • 


89 


86 


83 


Science Lab 


10 


20 


to 


66 


47 


48 


to 


76 


67 


23.7 


6.8 


90 


88 


82 


Conversation Topics 


10 


05 


to 


59 


25 


36 


to 


49 


48 


25.6 


5.4 


80 


74 


78 


Conver:it1on Information 


6 


20 


to 


55 


36 


45 


to 


64 


58 


14.7 


3.1 


81 


100 


84 


Scales - 1983 Edition 






























Science/Technology 


12 


x4 


to 


80 


30 


47 


to 


68 


56 


21.8 


7.1 


88 


mm 


m m 


Conversation Topics 


12 




to 


66 


26 


27 


to 


61 


49 


29.6 


5.4 


81 


62 


80 
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their respwRses accui'ately. Put another way, if one knew that a 
student "engaged frequently" in four of the activities, one would 
also know, given a perfect scale, exactly which four activities 
they were. A scale, defined in this way, means that there' is a 
.coherent and hicrarchial universe of content. The reproducibility 
coefficients in the table indicate that the quality df effort 
measures are close to the criterion for a good scale. It is 
partly because the activities in each measure have ^'"operttes— — - 
similar to a "scale" that the reliability of scores is so 
satisfactory. 

The .1983 revised edition of the College Student Experiences 
questionnaire no longeir included the Science Laboratory scale, and 
in the Conversation Topics scale one item was modified and two new 
ones were added. Psychometric data for the neW Science/Technology 
scale, and the revised Conversation Topics scale have been added 
to Table 1. Tor these 1983 scales N=2299. No factor anilysis and 
reproducibility data are shown for the Science/Technology scale • 
because it, by intent, consists of three different topics—science 
principles, science laboratory activities, and computers— and no 
single hierarchy was sought. The revised Conversation Topics 
scale has some item intercorrelations that are too low, and also 
now shows only 62% of the variance in the first factor. This 
factor consists of the topics related to major social issues and 
Intellectual subjects. 

Table 2 shows the distribution of scores on each of the 
quality of effort scales of approximately 10,000 students from 40 
colleges, obtained during the three-year period 1979 through 1981. 
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TABLE 2 



DISTRIBUTION OF SCORES ON THE. QUALITY OF EFFORT SCALES 
(N - 10»156 students in 40 colleges, accumulated from .1979 through 1981) 



3 



SCORES 


. LIB 


FAC 


1 ■ ■ 

COURSE 


UNION 


ATH 


CLUBS . 


36-40 


1 


1 


• 

12 


2 


3 


5 


31-35 


3 


4 , 


30 


5 


5 


6 


2-6^M - -- 


- — 14--- 


-----il — ^ 


' 36" 


—--14- 


g 


11 


21-25 


. ' 24 


24 


19 


27 


15. 


19 




- 38 - 


- -39 


■--■-3 


• 31 


25 


29 


11-15 

* 


18 


20 


0 


19 


31 


27 


10 (never) 


2 


1 


0 


2 


12 


3 " 




(100%) 


(100%) 


(100%) 


(100%) . 


(100%) 


(100%) 


Mean 


19 . 9 


20. 1 




Ofi Q 


its. .9 


on o 


Sigma 


5. 1 




0. U 


f C 1 

' D . i. 


f . 1 


/. i 


- 

SCORES 


WRITE 


PERS EXP 


ST ACQ 


DORM F/S 


SCI LAB 


CONV TPS 


36-40 , 


4 


2 


6 


7 


4 


5 


31-35 


13 


7 


12 


13 


10 


13 ' 


26-30 


26 


17 


24 


25 


20 


32 


21-25 


32 


• 30 


29 


29 


28 


34 


16-20 


20 


32 


23 


19 


21 


13 


11-15 


5 


13 


5 


5 


6 


3 


10 (never) 


0 


1 


1 


2 


11 


0 


(100%) 


(100%) 


(100%) 


(100%) 


(100%) 


(100%) « 


Mean 


24.7 


21.8 


24.6 


24.8 


. 22. 6 


25.7 


Sigma 


5.8 


5.8 


6.3 


6.5 


7.3 


5.4 



See Appendix for key to abbreviBtion of scale titles. 

■ > 
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TABLE 2 (continued) 



SCORES AMT SCORES CONV INFO 



43-48 


0 


22-24 


3 


J7-42 


2 


19-21 


8 


31-36 


6 


16-18 


26 


25-30 


15 


13-15 


36 


„_19--24 




10-12 ~ . 


23 


13-18 


39 


7-9 


3 


. 12 (never) 


6 


6 (never^ 


1 




(100%) 


(100%) 


(100%) 


Mean 


20.3 


Mean 


14.7 


Sigma 


6.2 


Sigma 


3.2 



1983 data for new or revised scales 
(N = 2,299 from 8 colleges) 



SCORES 


SCI/TECH 


CONV TPS 


43-49 


1 


2 


37-42 


3 


9 


31-36 


8 


30 


25-30 


17 


38 


19-24 


29 


18 


13- J8 


37 


3 


12 (never) 


5 


0 




(100%) 


(100%) 


Mean 


21.6 


29.1 


Sigma 


7.2 


5.8 
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When data for eleven colleges In 1982 were added, no percentage at 
any point in the distributions was changed by more than two points, 
and nearly two-thirds of the new percents were identical with the 
ones shown in Table 2. The 40 colleges included 8 large doctoral 
granting universities, 14 comprehensive colleges and universities 
about equally divided between public and private, and 18 liberal 
arts colleges about equally divided between Types I and II in the . 
Carnegie Commission'^s classification system. Apparently, by the 
end of the three-year period/the distribution of scores. had become 
quite stable. 

Two of the distributions (Athletic and Recreational Facilities, 
and Art, Music, Theater) show scores concentrated at the low end 
of the scale, owing mainly to the fact that many students have 
little or ho contact with those facilities. One distribution 
(Course Learning) has its scores cor cent rated toward the high end 
of the scale,, indicating that many students engage, frequently in 
most of the course learning activities. The other scales have 
distributions of more normal shape. 

t 

The SCOT- distributions on the new scales for the 1983 second 
edition are based on 2,299 students from 2 doctoral universities, 
1 compretjensive college, and 5 liberal arts colleges. These 
distributions may well change when a larger and more eclectic data 
base has been attained. The scores at the low end of the Science/ 
Technology scale are owing mainly to the lack of student activities 
involving use of c|)mputers, and the fact that some students have 
not had experienc^' in a science laboratory. 

34 
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Internal Relationships 

Another way to judge the confidence one might put in the 
various measures In the College Student Experiences questionnaire 
is to examine whether their relationship with other variables 
"makes sense." 

The first set of explorations that should "make sense" are 
ones between certain quality of effort scores and characteristics 
of the students' status in college. For example, some quality of 
effort scales are clearly concerned with academic and intellectual 
activities— use of the library, contacts with faculty members, 
course learning, writing, and science laboratory. One would 
surely expect or at least hope that these intellectual efforts 
would be greater among seniors than freshmen, would pay off in 
higher grades, be greater among students who aspired to continue 
their education beyond college, and who spent more time on their 
studies. Table 3 shows that these expectations are amply confirmed. 
Any difference between mean scores of 1.0 or more is statistically 
significant. On the scale measuring experience in writing there 
are no differences related to year in school, but there are the 
expected differences related to grades, aspirations, and time 
spent on academic work. Table 4 shows that on the various quality 
of effort scales related to the use of non-academic facilities on 
the campus and to the opportunities for students interpersonal 
relations and self-understanding-'-namely, the scales concerned 
with cultural facilities, clubs and organizations, the student 
union, athletic and recreation facilities, personal experiences, 
and student acquaintances--there are significant differences 
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TABLE 3 



COLLEGE STATUS CHARACTERISTICS 
RELATED TO ACADEMIC EFFORT 







Ouality of Effort Mean" Scores 




Colleae Status 


LIB 


FAC 


COURSE 


WRITE 


SCI LAB 


Year in College 












Freshman 


17.9 


17.8 


29.0 


24.7 


22.6 


Sophomore 


19.1 


18.6 


29.6 , 


24.5 


23.7 


Junior 


19.4 


19.2 


29.9 


24.0 


23.9 


Senior 


20.5 


20.3 


29.9 


24.6 


23.6 


uraaes 












B- or lower 


18. 5 


17.5 


28.1 


23.6 


■ OO 0 


B 


19.3 


19.0 


29.7 


24.6 


23.3 


B+ or higher 


1 Q O 


20.2 


30.7 


24.7 




Aspire to Adv. Degree 












No 


18.0 


17.9 


28.2 


23.3 


21.9 


Yes . 


19.8 


19.5 


30.1 


24.8 


24.2 


Hours on School - 












Related Activities 












20 or less 


17.8 


17.7 


27.8 


23.1 


20.9 


30 


19.1 


18.6 


29.8 


24.2 


23.0 


40 or more 


20.4 


20.3 


30.8 


25.3 


25.5 
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TABLE 4 



COLLEGE STATUS CHARACTERISTICS 
RELATED TO USE OF NON-ACADEMIC FACILITIES AND 
OPPORTUNITIES FOR PERSONAL AND INTERPERSONAL ASSOCIATION 



Quality of Effort Mean Scores 



College Status 


AMT 


CLUBS 


UNION 


ATH 


PERS EXP 


ST ACQ 


Have Lived on Campus 














No 


18.3 


15.9 


18.7 


15.4 


20.5 


21.9 


Yes 


21.4 


21.3 


22.0 


19.4 


22.8 


25.5 


Where Live Now 














With parents 


18.3 


16.4 


19.4 


16.0 


.20.7 


21.8 ^ 


Apt. away from 
campus 


18.9 


16. a 


. 18.6 


15.4 


20.7 


22.5 


Apt. near campus 


20.5 


19.0 


21.5 


18..0 


22.0 


24.1 


College housing 


21.7 


21.8 


22.0 


19.8 


22.8 


25.9 


Enrollment 














Part-time 


16.9 


14.0 


16.0 


14.1 


19.6 


20.3 


Ful 1-time 


20.1 


19.0 


20.6 


17.6 


21.8 


23.9 



> 



/ ■ 



1 
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between students who have lived on campus at some time and those 
who have never lived on campus, between those who now live in 
campus housing or near the campus and ones who liv^ farther away 
or with parents, and between full-time students and part-time 
students. Another comparison, in Table 5, shows the very sharp 
contrasts that are .found between science majors and humanities/ 
arts majors in the quality of effort scales related to those 
fields. Science majors are strikingly much more involved in the 
Science Lab scale activities than are the Humanities/Arts majors; 
and precisely the opposite is true of the activities related to 
cultural facilties and writing where the Humanities/Arts majors 
have much higher scores. These three tables present findings that 
are fully supportive of what some people in the field of measurement 
might call the concurrent or construct validity of the quality of 
effort scales. The results are congruent with prior research 
showing that residents are much more involved in Campus activities 
than are commuters. We also know from prior research that people 
who aspire to advanced degrees and who get high grades are usually 
more committed to taking advantage of the opportunities for learning 
and study and intellectual contact that college provides. And we 
would ^ost surely expect to find the differences between science 
and humanities/arts majors that are reported. All these relation- 
ships "make sense" and in that respect add confidence to what is 
being measured. ' * 

The Credibility of Self-Reports 

All questionnaire surveys are based on self-reports. What 
evidence 1s there that students' responses to the various items in 
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TABLE 5 

CONTRASTS IN QUALITY OF EFFORT BY MAJOR FIELDS 







Quality of Effort Mean Score 




M^jor Tield 


SCI LAB 


AMT 


WRITE 

• 


Sciences 


27.6 


19. 1 


23.7 


' Humanities/Art 


20.4 


, 25.0 


26.0 



the College Student. Experiences questionnaire are credible?' Can 
we believe what they say about their activities, their progress, 
their status in college, and about themselves? 

In survey research generally, the accuracy of answers depend 
on the clarity of questions, on whether the respondents have a 
good base in experience for answering the questions, on whether 
the form in which the answers are to be given is appropriate, and 
on whether the respondents regard the questions themselves as 
meriting a serious and thoughtful response. 

Several facts about the questionnaire lead us to the belief 
that all of the above conditions are met. In pre-testing many of 
the activities in the quality of effort scales we informally asked 
some students whether they had any trouble responding to them 
because of any lack of clarity. They said no. We tried to limit 
the activities to ones that were quite specific so that students 
would immediately know whether they had done them, and we asked 
th^m only to recall what they had done "during the current school 
year. " 
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• The form of response for the activities scales gave them a 
choice of indicating "never," "occasionally," ''often," and "very 
often." The meaning of "never" is self-evident. What one student 
means by "often" is not necessarily the same as what another 
student means. But for any individual student there is no doubt 
that "often" is more frequent than "occasionally," and that "very 
often" i^ more frequent that "often." . 

The items under the Estimate of Gains section of the ques- 
tionnaire are not so obviously explicit and specific as ti.e 
activities items, and students are asked to consider how much gain 
or progress they believe, they have made "in college up to now." 
At the same time, the topics are ones that are surely familiar to 
college students— topics such as writing well, thinking logically, 
the ability to find information they need, etc. Credibility in 
this crucial part of the questionnaire rests on whether their 
responses match what we know from other sources about student 
achievement, and on certain internal consistencies in the 
responses. We have already noted the sharp differences in quality 
of effort scores between science and humanities majors. We know 
from decades of data on objective achievement tests such as the 
Area Tests of the Graduate Record Examination, the advanced subject- 
matter tests of the GRE, and the major field tests of the Educational 
Testing Service that students make their highest scores on those 
tests, or parts of tests, that are most closely related to their 
major field, or to their "area of interest." Since we know this 
is objectively and extensively documented, the students' self- 
estimate of gains in the questionnaire should reflect that truth. 
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They do. Students whos'^ fnajor field Is science^f^prt the most 
ga^n in science- related goals, ^^fdt^^example, towar'^ the objective 
"understanding the nature, of science and experimentation," 84% of 
the science majors reported substantial progress, compared to 11^ 
and 15% respectively for, fine arts and humanities majors. Toward 
the objective "developing an understanding and enjoyment of art, 
music, and drama," 85% of the majors in fine arts reported sub- 
stantial gains. This percentage is three times greater than the 
percentage for science majors or social science. majors. Toward 
the objective "broadening your acquaintance and enjoyment of. 
literature," three-fourths of the humanities majors reported sub- 
stantial gains compared with one-fourth of the science majors. : 1 . 

For other objectives— such as "understanding other peoples- 
all sources of extfirnal evidence are sutrjective. By analogy with 

the correspondence tetweemnaj or fields and tooth reported gains '■ ~ 

and objectively measured gains, we can look for correspondence 
between reported experience and reported progress that "makes 
sense." For example^ students who make the most use of athletic 
and recreational facilities also report the most gain in "developing 
good health habits and physical fitness." Students who have the 
highest scores on the "Student Acquaintances" scale, and students 
who live in college housing, report the most progress toward the 
objective of "understanding other people and the ability to get 
along with different kinds of people." There are many other 
examples in the questionnaire responses of congruent or validating 
relationships between activities and achievement. Consequently, 
we can accept the self-reports of activities and the self-estimates 

of progress as broadly credible, valid, and true to the facts. 

. . ' *■ 
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Other observations also lend support for the belief that 
students' responses should be accepted as reliable and valid. 
This is evidence that they responded conscientiously to the questions 
The instructions for answering the questionnaire are to use a soft 
black lead pencil, to fill in the circles carefully, and to fill 
in an I.O. number that is printed at the ond of the questionnaire. 
Our estimates, from inspecting many questionnaires as they were 
turned in, are that 95% of the students filled out the entire 
questionnaire carefully and neatly. Also, we have examined the 
computer print-out far every item response in the 1983 second 
edition and have found that, wHh a couple of very small exceptions, 
never more than 2% of the students left an item blank. On the 
activity scales, and any other part of the questionnaire that 
produces a "score," the computer instructions are to compute a 
score only for those students who answered a12 the items in the 
scale. Again, with very minor exceptions, no more than 2% of the 
cases were left unscored. In short, in 95% to 98% and more of the 
students' responses we found no cause for concern about the care 
with which their answers were repdrted. 

Finally, we have been told by administrators at several 
colleges that quite a few students said they liked the question- 
naire, enjoyed filling it out, and felt they had learned something 
about themselves. Given the urual student attitudes toward ques- 
tionnaires, these favorable student comments suggest that they 
regarded the content of this questionnaire as meriting a serious 
and thoughtful response. 
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RESULTS 

Over the last few years, since the College Student Experiences 
questionnaire was first made available, many studies have been 
made of the questionnaire results. Perhaps the most Important 
line of Inquiry has been to assess what connections there may be 
between students' quality of effort and their attainment of 
important goals of education toward which their efforts were 
presumably directed. In the first section we noted that process 
variables were usually evaluated by the contribution they made to 
outcome or product variables. So, we begin this section on results 
by reporting those connections. Having then documented the overall 
significance of quality of effort, we turn to some studies of the 
diagnostic and differential -significance of quality of effort— the 
insight it gives to explaining and modifying the presumed benefits 
of "time on task;" of living on campus; of identifying the elements 
that contribute to the success or failure of, community college 
transfer students; of the significance of reading and writing 
activities in college education today; and of the very different 
activities and achievements associated with the students' major 
field of study. These capabilities of the quality of effort 
measures are further examined in a larger context— noting, for 
example, differences between types of institutions, differences 
among colleges that on most surface aspects appear to be very 
similar, and evidence of the uniqueness df a hitherto unstudied 
segment of higher education, namely, Bible colleges. And finally 
we proposie, and give evidence to support, an index of the quality 
of undergraduate education,, or more specificany the quality and 
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vitality of undergraduate student experience. Along the way, we 
shall Indulge In comments about the popular concept of "value < 
added," about the relations between gains and Initial status, 
about the use of the College Student Experiences questionnaire 'in 
Institutional research and In the se1f"-study aspect of accreditation 
review, and ultimately try to "add up" what we have Teamed about 
the quality and effectiveness of undergraduate education. We can 
say, In a preview of these results that quality of effort Is a 

f 

concept whose value Is predictive, diagnostic, and pervasive In 
understanding student learning and development in the college 
environment. 

The Prediction of Achievement 

o 

The first set of results, based on an analysis of 3,006 
student responses at 11 colleges and universities In 1979, Is, In 
many respects, the most dramatic. Subsequent analyses have merely 
confirmed these results. The question to be Answered Is this: 
given all the elements In the. questionna1re--students background 
characteristics, their status in college, their satisfaction with 
college, their assessment of the college environment, and their 
scores on the various quality of effort (QE) scales--what best 
predicts their achievement with respect to the list of goals of 
higher education? In the 1979 edition of the questionnaire there 
was a list of 18 goals. A factor analysis of these go^ls indicated 
that they could be grouped Into four broad clusters as follows: 

1. Personal/Social Development 

Developing your own values and ethical stariards 

Understanding yourself- -your abilities, interests, and 
personality 1, - 



Understanding other people and the ability to get 
along with different kinds of people , 

Ability to function as a team member 

Developing good health habits and physical fitness 

2. Genera), -Education, Literature and Arts 

Gaining a broad general education about different 
fields of knowledge 

Developing an understanding and enjoyment of art, 
I "Tflusic, and drama . 

Broadening your acquaintence and enjoyment of literature 

Writing clearly and effectively 

Becoming aware of different philosophies, cultures, 
and ways of 1 1 f e 

3. Intellectual Skills 

Ability to think analytically and logically 

Abilit^' to put ideas together, to see relationships, 

similarities, and differences tetween-ideas — --^ — ^- 



Ability to learn on your own, pursue ideas, and find 
information you need 

Acquirinjj background and specialization for further 
education in some professional, scientific, or 
scholarly field . 

4. Understanding Science 

Understanding the nature of science and experimentation 

Understanding scientific and technical developments 
and their applications in society 

» 

Quantitative thinking— understanding probabilities, 
proportions, etc. 

Students reported progress toward each of the objectives in th^sse 

four groupings were added up to give an estimate of progress 

toward the more general objective defined by the group category. 

These four categories of achievement were then used asxriteria of 
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attainment, to be predicted or explained by all the information we 
have aboyt students, environments, and quality of effort. The 
statistical procedure is called stepwise multiple regression. 
This simply means that the computer program first idenv,fies the 
variable that has the largest relationship with the criterion, 
then the variable that has the next largest relationship, etc., 
until adding more variables contributes little or nothing more 
(less than 1^)' toward accounting for the performance on tbe criterion. 

technical tables giving the results of the factor analysis, 
and of the regressions are in the Appendix—Appendix tables 1 and .: 
2. 

What the figures in those tables show is quite clear. In 
relation to every one of the four main categories of achievement, 
one or more of the quality of effort scales (QE) makes the greatest 
contribution toward explaining thai achievement. The numbers 
under the column labelled are really percentages—that is, they 
show the percent of the variance on the criterion that is "accounted 
for," or more simply, "when you know these things this is how much 
of the result you have been able to explain or predict." 

The best predictors of s:tudents' progress toward personal /social 
development objectives are the quality of effort scales dealing 
with activities intended to promote self -understanding and with 
the use of athletic and lecreation facilties. The best predictor 
of students' progress in acquiring intellectual skills is the 
quality of effort they devote to course learning activities. The 
best predictors of progress toward goals related to general education 
literature and arts are the quality of effort students Invest in 



using the cultural facilities on campus for art, music, and drama, 
and the quality of effort they put Into writing activities. The 
best predictor of students' progress In understanding science Is 
the quality of effort they put Into using science laboratory 
facilities^. ^_ 

Another way to highlight the contribution that quality of 
effort makes In predicting achievement is to put all the variables 
into the computer in a predetermined sequence: first, put in all 
the students' background or status variables; second, put in all 
the college status variables; third, put in all the environment 
ratings; and finally, after all these commonly utilized variables 
have contributed as much as they can to explaining achievement, 
put in the quality of effort variables to see whether they add. 
anything to explaining the achievement. These results are shown 
in Appendix Table 3. 

Before entering the quality of effort measures, one can 
account for somewhere between 24% and 36% of the result on the 
criterion. This is almost exactly what many past studies have 
shown. When the quality of effort measures are added, one can now 
explain from 39% to 47% of the performance on the criterion—a 
substantial Increase in our understanding, from 10 to 15 percentage 
points better than past research has typically been able to explain. 

The results just described lead to a very significan* conclusion, 
one that differs from much prior research which has held that 
student characteristics and family background are the most important 
determinants of achievement. The new conclusion Is this: granted 
the importance of all the elements that Influence who goes where 
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to college, once the students get there what counts most is not 
who they are or where they are but what they do . Prior research 



has not included what turns out to be the most influential 
variable— the quality of effort that students themselves invest in 
using the facilities and opportunities fgr learning and develop- . 
ment that exisi in the college setting. Now that "quality of 
effort" has bejsn included, better explanations and new conclusions 
•emerge. ./ 

The Prediction of Achievement Reaffirmed 

Because the introduction of qualify of effort into equations 
for predicting achievement has clearly modified the generaliza- 
tions made from prior research, one needs to be sure that the new 
generalizations are reasonably stable. When the revised second 
edition questionnaires from 2299 students at 8 colleges had been 
filled out, we repeated all of the analyses we have just reported- 
results obtained from different students, at different colleges, 
at a different time, and with a questionnaire that contained a few 
slightly different items. Do we still get- the same results? The 
answer is yes, with a few minor adjustments. These analyses were 
made by Karen Lefever. The previous analyses of the 1979 data had 
been made by Jack Friedlander. 

The first analysis we had made, shown in Appendix Table 1, 
was a factor analysis of the estimate of gains. In the second 
edition of the questionnaire there were 21 statements of objective 
One of the original- 18 objectives, "Vocational training—acquirin 
knowledge and skills applicable to a career," was now divided int 
two objectives, one focused on specific job training, and one i 
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focused on broader career relevance. Another of the objectives 
listed in the 1979 edition was now also split into two more clearly 
defined statements: the goal "understanding scientific and technical 
developments and their applications in society" was expanded in 
the 1983 edition into two goals— one focused on understanding new 
developments; and one focused on awareness of the consequences of 
new applications. Alsa, in the 1983 edition there was a new 
objective: "Acquiring familiarity with the use of computers," 

The factor analysis of the 21 items included in the Estimate 
of Gains sectioh of the revised questionnaire is reported in 
Appendix Tat>le 4. There now emerges a factor we have called 
"Vocation" consisting of three items. In the 1979 questionnaire 
there had only been one goal statement about vocation and it did 
not fit into any of the four goal categories in the 1979 factor 
analysis. Except for this new factor, which we had hoped would 
emerge, the factor analysis results of the new data are nearly 
identical with the previous analysis. For the revised second 
edition questionnaire, the major categories of goals are as follows:,^ 

1. Personal and Social Developmeht 

Understanding other people and the ability to get 
along with different kinds of people 

Understanding yourself--your abilities, interests, and 
personality 

Developing your own values and ethical standards 

Ability to function as a team member 

Developing good health habits and physical fitness 

2. General Education, Literature and Arts 

Broadening your acquaintance and enjoyment of 
literature. 
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4. 



Writing clearly and effectively 

Developing an understanding and enjoyment of art, 
music, and drama 

Becoming aware of different philosophies, cultures, 
and ways of life 

Gaining a broad general education about different 
fields of knowledge 

Intellectual Skills 

Ability to think analytically and logically 

Ability to put ideas together, to see relationships, 
similarities, and differences between ideas 

Quanti tati ve thi nki ng—understandi ng' probabi 1 i ti es , 
proportions, etc. 

Ability to learn on your own, pursue ideas, and find 



Understanding Science/Technology 

4 . 

Understanding new scientific and technical developments 

Understanding the nature of science and experimentation 

Becoming aware of the consequences (benefits/hazards/ 
dangers/values) of new applications in science 
and technology 

Acquiring familiarity with the use of computers 
Vocation 

Vocational training"acquiring knowledge and skills 
applicable to a specific job or type of work 

Gaining a range of information that may be relevant to 
a career 

Acquiring background and specialization for further 
education in some professional, scientific, or 
scholarly field 

ults of the step-wise multiple regression analysis of all 

s in the questionnaire in relation to each of the above 

egories of goals are in Appendix Tables 5 and 6. Again, 



information you need 
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scores on various quality of effort scales emerge as major con- 
tributors to students' achievement. For example, toward the broad 
goal of personal/ social development, students' quality of effort 
on the Dormitory, Fraternity/Sorority scale makes the largest 
contribution, with the quality of effort scales related to Athletic 
and Recreation facilities, and to Personal Experiences also being 
influential. Toward the cluster of goals related to general 
education, literature and arts,, the biggest contribution comes 
from students' scores on the quality of effort scale in using the 
campus facilities of art, music, and theatre. The top contributor 
to students' progress toward understanding science and tephnology 
is the quality of effort scale labeled Science/Technology. In the 
developipent of intellectual skills, having an environment character- 
ized by a strong emphasis on being' critical , evaluative, and 
/analytical, heads the list of influential variables, followed by 
the quality of effort scales regarding Science/Technology, and the 
informational and persuasive level$ of student conversations. For 
the outcome labeled vocation, no spjecifically related quality of 
effort scale had been developed. For this outcome the most influ- 
ential variables were an environment with strong emphasis on the 
development of vocational and occupational competence and a strong 
emphasis on the personel relevance and practical value of the 
students courses. 

When all the variables were put into the regression equation 
in a predetermined order--first, student characteristics; second, 
college status variables; third, ratings of the college environment; 
and finally, quality of effort--the results again show that quality 
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of effort makes a significant added contribution to explaining the 

outcomes. For the personal/social development goals, increases 

from .34 to .46 when quality of effort is added to the predictors; 

for the general education goals, the increase in i.s frdm .48 to. 

.55; for intellectual skills the increase is from .37 to .46; for 

science goals the increase is from .34 to .59; and for vocation 

2 

the increase in R is from .45 to .50. 

For the second edition '1983 results, we have also summarized 
the simple and direct correlations of all v_ariables with each of 
the clusters of gains. This information, of course, overlaps with 

the regression analysis results; but it provides a little different 

' i 

and more detailed understanding of how the data fit together. In 
the lists below, aU correlation of .30 or higher are shown. (See 
Appendix for an explanation of ttte abbreviations.) 

Correlations with Personal/Social Development Gains 



QE:Dorm F/S .40 

QE:Conv Tps .39 

QE:Pers Exp .38 

QE:Conv Info .34 

QE: Clubs .33 

QE:St Acq ' .32 

ENV: Student , .32 

ENV:Rel .31 

QEiAthl s .31 

QE: Course .31 
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"Correlations with Intellectual Skills Gains 

ENV:Crit /' .43 

QE: Course .40 

QE:Conv Info / .38 

QE:Sci/Tech I - - - - 3^ - 

ENViSchbl — I .31 

Correlations with Geneifal Education, Literature and Arts 



Gains 



QE:AMT / 


.49 


QE:.Conv Tps/ 


.41 


ENV;Crit / 


.38 


Read Texts/ 


.36 


Q£:St Acq/ 


.36 


QE:Conv l/nfo 


.35 


QE:Fac / 


.35 


QE: Write/ 


.34 


QE: Course 


.33 


ENV:Estii 


.33 


Write papers 


.30 



Correlations with Understanding Science/Technology Gains 



QE:Sc^7Tec .71 
Correlations w^th Vocation Gains 

ENY:/Voc • .43 



ENV-Rel .41 



The frequency with which QE measures appear in these lists is 
very obvious. In addition to the 29 correlations identified 
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, above, there are 44 others that fall between the range of .20 
and .29, making a total of 73. When all of these are classified 
as to their origin or type, the results are as follows: 

None are Student Characteristics variables 
15 are College Status variables 
22 are College Environment variables 
and 36 are Quality of Effort variables. 
This again reinforces the conclusion that, in the prediction 
of achievement in college, what counts most is not who you are. 
(student characteristics variables), or where you are (college 
status and college environment variables), but what you do (quality 
of effort variables). 

The Prediction of Satisfaction with College 

If students don't like what they are doing, and would rather 
be somewhere else than in the college they are presently attending, 
the chances are that they are wasting their time and not taking 
advantage of the opportunities available to them. Enthusiasm, 
satisfaction^ and a sense of worthwhileness' are personal feelings 
that contribute to productivity and accomplishment. Fortunately, 
most students in most colleges like being there. The College 
Student Experiences questionnaire produces an index of satisfaction 
based on responses to the following questions: 
Mow well do you like college? 

I am enthusiastic about it 

I like it 

I am more or less neutral about it 
1 don't like it 
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If yo^ Gould start over again, would you go to the same 
college you are now attending? 

Yes. definitely 

Probably yes 

Probably no 

No, definitely 

By giving 4 points to the most favorable response to each question, 
3 points to the next most favorable response, etc. and adding the 
points, one gets a score, or satisfaction Index, ranging from 2 to 
8. The meaning of scores 8 or 7 at the most favorable end of the 
scale, and of scores of 2 or 3 at the most unfavorable end are 
clear. A score of 6 is nearly always obtained by students who 
said "I like it" to the first question, and "Probably yes" to |he 
second question, Anyone getting a score of 5 or lowerftiust have 
answered at least one question negatively or neutrally. 

An analysis of responses from a cross-section of UCLA under- 
graduates in 1979 produced the following distribution. 
Satisfaction score 

8 points 22% very satisfied 

7 points 27% 

6 points 28% satisfied 

5 points 13% 

4 points 7% neutral to negative 

3 points 2% 

2 points 1%^ 

In the UCLA study of satisfaction we divided the students 
into three groups, as defined above—very satisfied, satisfied, 
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and neutral to negative—and then compared the responses of these 
three groups with respect to the quality of .effort scores on all 
14 aspects of the college experience, their characterizations of 
the college environment on all of the environment ratinjs, and 
their ratings of progress toward all of the objectives. On every 
quality of effort scale, and pn every characterization of the 
environment, and on reported progress toward every objective, thp 
highest (most favorable) mean scores were made by the students wkjo 
were "very satisfied" with UCLA, the next best scores were made b/ 
students who were "satisfied," and the lowest scores were made by 
those who were "neutral to negative." There were no exceptions to 
this pattern. 

Students who are most satisfied with college put the most 
into it and get the most out of it. Using satisfaction with 
college as the criterion, and then determining which variables of 
all the ones included in the questionnaire have the highest rela- 
tionship to that criterion (best predict or best explain it), the 
^ two most i-nfluential variables were as follows: first, students' 
gains in the group of objectives we*have described as intellectual 
skills, and second, environment in which relationships among 
students were characterized as friendly and supportive. So, when 
students are very satisfied they believe they are developing their 
intellectual powers, and find the environment to be friendly and 
supportive. Since we don't really know the direction of these 
relationships we can also state the generalization another way: 
when students are making progress in the development of their 
intellectual po)rfers, and when their experience in the environment 
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1s characterized by friendly and supportive relationships with 
other students, they are very satisfied with college. 

Since having "satisfied" students is surely a desirable 
condition, further exploration of what contributes to satisfaction 
more generally, not just at UCLA, was conducted. It is possible, 
for example, that student satisfaction at research oriented univer- 
sities may result from a different combination of conditions than 
at other types of colleges and universities. 

To examine this possibility, Oscar Porter ran stepwise multiple 
regression analyses of all the questionnaire elements against the 
criterion of satisfaction (the satisfaction index score) separately 
for five different types of institutions—doctoral universities, 
public comprehensives, private comprehensives , liberal arts type 
I, and liberal arts type II.. The result of these regression 
analyses are shown in Appendix Table 7. In all types of colleges, 
the single most important contributor to students' satisfaction was 
an environment described as friendly, supportive, helpful etc.-- 
most commonly the supportive relations among students, but also in 
some cases the helpful, encouraging relationships with faculty 
members, or the" flexible, considerate style of the college's 
operations. Also contributing to satisfaction was an environment 
strong in Its emphasis on intellectual qualities--whether academic, 
scholarly, or critical, analytical. Parallel to this intellectual 
emphasis are the students' belief that they have gained In acquiring 
a broad general education, and, in all types of institutions that 
they are making good grades or at least not making poor ones. 
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The only quality of effort scales that entered into the 
regression equations were ones that involved personal/social 
relationships—Dormitory, Fraternity/Sorority, Student Acquaintences, 
Clubs and Organizations, Athletic and Recreation Facilities, 
Student Union, Conversation Topics, and, the one exception to this 
list, Library. 

By noting alj the variables in the .regressions that had a 
simple direct correlation with the Satisfaction Index of .20 or 
higher, we can seje with additional clarity the common sources of 
studeni satisfaction with college. Of the 55 variables listed in 
Appendix Table 7, 32 had correlations of .20 or higher with the 
Satisfaction Index. These 32 correlations are identified in 
Appendix Table 8. The categories or sources of these elements 
throw further light on what contributes to satisfaction. Of the 
32 correlations listed, 17 are ratings of the ENVIRONMENT (with 12 
of those referring to the supportlveness of personal relationships 
within the environment); 9 are estimates of GAIN or progress 
toward important goals (acquiring background and specialization 
for advanced education, breadth of knowledge, and self-understanding); 
and 6 are QUALITY OF EFFORT topics (all related to group facilities 
and interpersonal relationships). None of the 32 elements came 
from the College Status, or the Student Characteristics categories. 
The Diagnostic Value of Quality of Effort 

Quality of effort, introduced into equations in predicting 
students success in college, has significantly modified previous 
generalizations about what best accounts for achievement. The 
predictive value of quality of effort is more powerful than the 
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predictive, value of family background, racial or ethnic identifica- 
tion, age, sex, marital status, ^^or^^a^ous characteristics of the 
college environment. There is, in addition to this predictive 
significance, a diagnostic value in measuring quality of effort 
which also leads to some modifications in the conclusions from 
prior research. Examples of these refinements are described next. 

One of the variables that has been widely introduced in 
educational research, especially as related to achievement in \ 
elementary and secondary school. Is variable called "time on 
task." For example, a high school math course may be scheduled 
for one hour a day, five days a week, for 15 weeks; but from " 
observations of what really occurs during the class period one 
discovers that the actual time spent on the task (math work) has 
not been 75 hours, but, in some cases, no more than 50 hours, 
owing to various interruptions, digressions, etc., whereas in 
other cases as much as 90X of the time was actually spent "on 
task." Time on task is a- much better predictor of students' 
achievement than time allocated to the subject in the curriculum. 

In our research with the College Student Experiences question- 
naire we do not have any literal (direct observation) counterpart 
to time on task, but we have made comparisons between time spent^. 
and quality of time spent. These comparisons have been made by 
Jack Friedlander. For these comparisons, two definitions are 
similar to the idea of time on task. One is how long the students 
have been in college; the other is how many hours a week' the 
students usually spend on activities related their school work. 
The analyses confirm the iirportance of time, but also the greater 
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importance of effort. It's true, for example, that gains on the 
outcome measures related to intellectual skills and t6 general 
education are related to how long one has been in college—the 



quality of effort scores with respect to intellectual/ academic 
experiences (course learning, library, contacts with faculty, and 
writing) are above average report greater gains in intellectual 
skills and in general education than juniors or seniors whose 
quality of effort scores are below average. It is also true that 
sheer time spent on academic work (number of hours a week) is 
related to pro(Sress toward objectives related to general education, 
to Intellectual skills, and to grades. But the whole truth is 
that students who spend a lot of time at a low (below average) 
level of quality make less progress than students who spend fewer 
hours at a high (above average) level of quality; and students who 
spend about 40 hours a week of high quality effort get better 
grades than students who spend 50 or more hours of low quality of 
effort. Compared to quality effort, time spent (years in college) 
and time on task (hours spent on academic wgrk^ are relatively 
weak explanations. 

These relationships between time, quality of effort, grades, 
and gains are documented in Tables 6 and 7, based pn students from 
30 colleges and universities. . / 

Living on campus has been found by many researchers to be 
positively reVated to satisfaction, retention, and benefits of 
■ various kinds (both intellectual and social). In a study of 



gains reported by seniors are significantly greater than the gains 
reported by fres^hmen. But the whole truth is that freshmen whose 
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TABLE 6 



LENGTH OF TIME IN COLLEGE RELATED TO GRADES AND GAINS 
AS MODIFIED BY QUALITY OF EFFORT 



Year in College and Quality 
of Academic/Intellectual 
In Effort 



Grades 



Gains In 
Intellectual Skills 



Gains In 
General Education 



Freshmen 
Low QE 
High QE 


2.9 


2.8 
3.1 


10.6 " 


9.8 
11.5 


11.7 


10.8 
12.9 


Sophomores 
Low QE 
High QE 


3.1 


3.0 
3.2 


Its 


10.7 
12.3 


12.3 


11.1 
13,5 


Juniors 
Low QE 
High QE 


3.2 


3.0 
3.4 


11.8 


10.9 
12.5 


12.3 


11.3 

13.2- 


Seniors 
Low QE 
High QE 


3.3 


3.1 

. 3.4 


12.1 


11.3 
12.7 


12.5 


11.4 
13.3 



N = 7700 
Note: 



Grade point averages are 2.0 = C; 3.0 = B; 4.0 -A. Mean scores on the 
Intellectual Skills and General Education objectives that are different 
from one another by .3 or more are statistically significant. 
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TABLE 7 



HOURS PER WEEK ON ACADEMIC ACTIVITIES RELATED TO GRADES 
AND GAINS AS MODIFIED BY QUALITY OF EFFORT 



Year in College and Quality . n 

of Academic/Intellectual Gains in Gains in 

in Effort Grades Intellectual Skills General Education 



About 20 hours or less 
Low QE 
High QE 


2.9 


2.8 
3.0 


10.6 


10.0 
11.8 


11.2 


10.5 
12.6 


About 30 Hours 
Low QE 
High QE 


3.0 


2.9 
3.1 


11.3 


10.6 
12.1 


12.2 


11.3 
13.2 


About 40 hours 
Low QE 
High QE 


3.3 


3.2 
3.4 


11.9 


11.1 
12.4 


12.7 


11.6 
13.4 


About 50 hours or more 
Low QE 
High QE 


3.5 


3.3 
3.6 


12.3 


11.5 
12.8 


12.6 


11.2 
13.3 
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campus vs off-campus residents at four large doctoral granting 
universities, as well as a special stu(ty of UCLA students, the 
data from the College Student Experiences questionnaire are 
generally supportive of prior research on this topic—but with one 
diagnostic refinement when quality of effort Is considered. The 
analyses of these data were made by Mary Beth Snyder. The student 
respondents from the four universities differed In several ways- 
such as the proportion of freshmen vs upperclassmen, etc. , but the 
differences between those who lived on campus (dormitory or 
fraternity/sorority) and those who did not were so large along 
certain c(1men$1ons that other variations between the groups could 
not have reversed the conclusions. Moreover, with respect to the 
quality of effort students Invested In using group facilities and 
opportunities for personal and social development and with gains 
related to personal and social development, and with overall 
satisfaction with college, the differences are totally consistent 
in all four institutions, except for one item at one institution. 

The biggest differences between campus and off-campus groups 
were in the estimated gains in personal and social development as 
shown in Table 8. 

The much higher proportion of campus residents claiming sub- 
stantial gain in these respects is parallel to the much higher 
percentage of campus resident > who score above a certain level in 
the quality of effort scales related to those gains as shown in 
Table 9. 

/ 

The other main difference between campus and off-campus / 
residents is in the former's perception of the environment as 
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TABLE 8 

COMPARISONS BETWEEN CAMPUS AND OFF-CAMPUS RESIDENTS IN 
GAINS RELATED TO PERSONAL AND SOCIAL DEVELOPMENT 



Values. Ethics 



Percent Reporting Substantial Gain 
Und Self Und Others 



Team 



Health 





Campus 


Off-Campus 


Campus 


Off- Campus 


Campus 


Off-Campus 


Campus 


Off- Campus 


Campus 


Off-Caropus 


University A 


57^ 


51 


73 


67 


82 


60 


52 


32 


54 


26 


University B 


79* 


5^ 


82 


69 


87 


51 


59 


26 


43 


: -:32V 


University C 


66 


55 


73 


59 


79 


62 


39 


21 


35 


29 


University D 


71 


59 


77 


65 


82 


68 


60 


42 


56 


35 



as 
o 



TABLE 9 

COMPARISONS BETWEEN CAMPUS AND OFF-CAMPUS RESIDENTS IN 
QUALITY OF EFFORT RELATED TO PERSONAL AND SOCIAL ACTIVITIES 









Percent Scoring Above a Defined Level of QE 










QE: 


PERS EXP 


QE: 


ST ACQ QE: CLUBS 


QE: 


ATHL 


QE: 


CONV TPS 




Campus 


Off-Campus 


Campus 


Off-Campus Campus Off-Campus 


Campus 


Off-Campus 


Campus 


Off-Campus 


University A 


62 


47 


44 


24 36 22 


34 


15 


54 


44 


University B 


62 


43 


48 


23 33 11 


38 


28 


57 


(42 ( 


University C 


55 


45 


45 


30 25 26 


34 


30 


59 


45 


University D 


55 


43 


40 


26 47 16 


46 


22 


56 


46 



64 



ERIC 



having a supportive and friendly relationship among students to a 
much greater extent. 

Other variables— such as quality of effort in academic, 
intellectual topics, and gains with respect to intellectual skills 
and general education, and in other characterizations of the 
environment—did not show consistently large differences between 
campus and off -campus groups; and, in fact, residence seemed to be 
neutral rather than positive in its influence. 

The mere fact of living on campus has some influence on 
students' progress toward goals of personal and social development 
and the quality of effort they put into various opportunities for 
group associations. But this is. not the whole truth. On the 
scale measuring the quality of effort students who live in a group 
setting on campus (QE: Dorm, F/S) put into ijsing those opportuni- . 
ties, some students have very high scores and others have very low 
scores. Some students, despite living in a group setting take 
little advantage of it. On those objectives most related to 
personal and social development, students whose participation in 
the residence unit activities ranked them in the lowest third of 
the scores on that scale gained no more than students living at 
home with their parents. These relatively uninvolved campus 
residents might as well have stayed at home. In this respect, the 
quality of effort scale permits a more refined and diagnostic 
understanding of the general value of living on campus, for the 
whole truth is that the benefit depends partly on what they do, 
not merely on where they live. 
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A third special study illustrating the diagnostic value of 
quality of effort was a local study at UCLA by Juan Lara of community 
college transfer students, comparing .the quality of effort at UCLA 
of transfer students who had persisted and those who had dropped 
out, and also the quality of effort of these two groups when they 
were in community college. The academic quality of effort scales 
were used in this study—course learning, library, writing, and 
faculty contacts. The population included all community college 
transfers to UCLA in the fall of 1977. Sixty one percent of them 
were located and responded to a questionnaire in the spring of 
1979—824 respondents were still enrolled at UCLA and 312 had 
dropped out. The students indicated how often they had engaged in 
the various activities at UCLA and also how often they had engaged 
in those same activities wf-cn they were in community college. 
Other parts of the questionnaire asked about some environment 
characteristics at community colleges and at UCLA, apd progress 
toward certain objectives at community college and at UCLA. On 
the quality of effort scales for library, writing,' and couirse 
learning, the scores of both the dropouts and the persisters were 
higher at UCLA than they had been at the community college; but 
the difference was much greater for persisters than for dropouts. 
In other words, the dropouts had increased their quality of effort 
somewhat, but not nearly enough and not nearly as hiyh as the per- . 
sisters. For example, on the Course Learning scale, the percent 
scoring 26 or higher on the scale was 55% at UCLA for those who 
had dropped out compared with 37% when they were in community 
college. In contrast, the corresponding percentages for those who 
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were persisters at UCLA was 50% when they were In community college, 
increasing to 80% at UCLA. On the library scale the persisters 
had 65% scoring 21 points or above at UCLA, compared with 31% for 
the dropouts. For both groups the percentages represented an 
increase over what their quality of effort in library use had been 
at the community college— the dropouts Having increased from 16% 
to 31% and the persisters increasing from 28% to 65%. 

- ■ ■ 

These differences between community college and whe university 
are also reflected in students' ratings of progress toward important 
objectives. In community college, less than a third of the transfer 
students felt they had made very much or quite a bit of progress 
toward the objective of ability to think analytically and logically 
(34% among those who subsequently persisted at UCLA and 27% among 
those who" subsequently dropped out). At UCLA, among those who 
persisted, 85% claimed very much or quite a bit of gain, comp.. red 
with 46% amcng those who had dropped out. 

From these examples, and from many others like them in the 
complete study, two generalizations can be made: first, the 
quality of academic effort needed for persistence at the university 
was much higher than the quality of effort needed at the community 
college to become eligible for transfer; and second, Ci)mpared with 
the students who later dropped out, the students who were successful 
at the university had not only made a much larger increase in 
their prior quality of effort but also had reached a much higher ^ 
absolute level. ^ 

This study had a major influence on the enactment of revised 

policies for the admission of transfer students; and for the 

1 
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provision of resources to assist those who may need help to succeed 
at UCLA. 

The Dltferantial Value of Quality of Effort 

Cal leges differ from one another in many ways-- in size» 
affluence, programs, selectivity, etc. --and the question now 
raised is whether and how they might also differ from one another 
in the quality of effort of their students. We do not, at this 
point, consider whether college A is different from college B, tout 
rather in a broader context whether certain types of colleges 
differ in some systematic way from othef types--not in ways we 
already know they differ, but in wayfe revealed by their students' 
responses to the College Student Experiences questionnaire. Since 
the unique feature of the College Student Experiences questionnaire 
is its measurement of students' quality of effort in using various 
facilities and opportunities for learning- and development, the 
first concern is in what differences there are in that respect. 

There are 14 quality o!" effort scales. Considering the 
results from 14,615 students at 62 college.s, collected over the 
four-year period from 1979 through 1982— doctoral granting univer- 
sities (DU), comprehensive colleges and. universities, public (PUB • 
COMP), and private PRI COMP), liberal arts colleges, types I and 
11 based on selectivity (LA I and LA II),. and Bible colleges— what 
differences are there in students' quality of effort at those 
institutions? The data from Bible colleges will be considered 
separately. The first comparisons, then, are between the other , 
five types of colleges and universities. 
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Differences in mean scores on the quality of effort scales of 

1.00 or more are always statistically significant, given. the large 

number of cases in the present analyses. But even thicugh statisti- 

cally significant, such relatively small differences are piobably 

of little practical or visible import. So, we decided to focus 

mainly on differences in mean scores of 3.00 or greater. Of the 

1^ quality of effort scales there are eight in which differences 

^between institutional types are of this larger magnitude, and two 

others which differ by more than. 2.00, but less than 3.00. Of the 

10 differing by 2.00 on more, three ^e activities related to 

academic scholarly efforts, three concern personal and social 

\ X, _ 

activitiiBS, and four are in the use of various g|rdup facilities on.. 

the campus. In other words, between the five types of institu- 
tions, 10 of the 14 quality of effort ^ales reveal major differ- 
ences in student activities. These differences include all the 
primary aspects of college experience--academic , personal/social, 
and the use of institutional facilities. 

On other quality of effort scales, the differences in scores 
between institutional types are small.' There are apparently few 
difference in student activities, no matter what sort of school 
one attends, with respect to Course Learning, Experience in Writing, 
Information in Conversations*, or Personal Experiences. 

What .emerges with striking clarity in comparing quality of 
effort differences between institutional types is the remarkable 
and almost universal superiority of the academically selective 
liberal arts colleges (LA I). They have the highest mean scores 
on the scales concerning Library, Experiences with Faculty, and 
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Science Laboratory; also on the scales concerned with Conversation 
Topics, Student Acquaintences, and Art, Music, Theater; and further 
on the activity scales regarding Clubs and Organizations, and 
Athletic and Recreation Facilities, ' 

These results at the selective liberal arts colleges are 
consistent with all prior research regarding instititional differ- 
ences. There are also results on the quality of effort scales 
that are consistent with all prior research on the. t:haracteri sties 
of student life at big schools vs. small schooU. For example. 



the small schools have higher quality of effort scores/on Experiences 
with Faculty, Personal Experiences, student Acquaintences^ Clubs 
and Organizations, Student Union, and^Athletic and Recreation 
Facilities. So, quality of effort is a variable that, on its own, 
reveals important institutional differences. 

The unique virtue of the selective liberal arts colleges is 
further documented by strong emphasis in their environment on 
scholarly qualities, on being analytical and critical, and on 
students' characterizations of the faculty as approachable, helpful, 
understanding, and encouraging. 

The combined features of high quality effort and the environ- 
mental emphasis on intellectuality and personal supportiveness in 
the selective liberal arts colleges is further enhanced, or perhaps 
helps to produce, gains toward many important objectives that are 
significantly greater than in other types of colleges. The unique 
quality of these liberal arts colleges is in the progress their 
students report in general education, literature, and arts. In 
breadth of knowlege, in becoming aware of different philosophies. 
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Cultures, and ways of life, in broadening their acquaintence and 

enjoyment of literature, and developing an understanding and 

rnjoym^nt of art, music, and drama, more students in the selective 

liberal arts colleges report substantial progress than do students 

in any other type of school. They also report most progress 

toward developing various in^ellectualT-^sWJJs. 

The objective toward whiph fey/fer students in the selective 

liberal arts colleges than in ^11 other types of colleges report 

substantial gain is "Vocational training— acquiring knowledge and 

skills applicable to a career." This finding, from the reports of 

students in college today, is of particular significance In relation 

to the frequently expressed belief that todays students are mainly 

"vocational" in their orientation and, presumably, not much interested 

in the "liberal arts." However, it is also the students in the 

selective liberal arts colleges who are most satisfied with college. 

In the large doctoral granting universities, most of them 

strongly research oriented, the relationships among students and 
J 

between students and faculty shown by the environment ratings are 
the lowest or least supportive of the five types of institutions. 
Students at the doctoral universities also-have the lowest mean 
score on the quality of effort scale regarding contacts with 
faculty members. Nevertheless, the percent of students at the 
doctoral universities who are "satisfied with college" is high-- 
higher than the percent at the more congenial and supportive 
liberal arts type II colleges or the private comprehensives. A 
friendly environment usually contributes to student satisfaction, 
but it does not assure it. 
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.The basic data, showing Institutional differences In quality 
of effort, environment ratings, and estimates of progress are in 
Appendix Tables 9, 10 and 11. 

To test still further the ability of the quality of effort 
measures to differentiate between colleges, Oscar Porter selected 
six colleges that appeared to be very similar to one another (all 
were highly selective liberal arts colleges of roughly similar 
size and affluence, and all, in their catalogues and brochures had 
very similar statements about their dedication to "liberal education," 
and their concern for Individual students). Yet colleges typically 
believe that they offer unique undergraduate experiences. Porter's 
dissertation research sought, among other things, to see whether 
variations in student effor't might provide a key to understanding 
institutional uniqueness, whether the assumption of uniqueness can 
be supported by quantitative data on how the students interact 
with th« institution's facilities and opportunities for their 
educatton. 

Composite data, as in comparisons between types of institutions, 
mask whatever differences there may be between individual institutions. 
On the 14 quality of effort scales there were four which showed 
large differences between these selective liberal arts colleges. 
Three were scales involving students' use of group facilities and 
associations--student union, athletic facilities, clubs and 
organizations--and the fourth wa$ the library scale. On the 
activities related to student personal and social development, and 
most of the activities ir»volving intellectual aspects of college 
life, differences between the institutions were generally small. 
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There clearly was some uniqueness which institutions could claifn, 
however. 

The activity "met your friends at the student union pr student 
center" was a frequent activity of 86% of the students at one 
college, and of 36% at another college. At one college 47% of the 
students said they had frequently played on an intramural team, 
compared with 5% at another college. At one college 45% of the 
students said they had frequently attended a meeting of a club, 
organization, or student government group; but at another college 
this was a frequent activity for 15% of the students. On the 
library scale, to take one more example, 42% of the students at 
one college frequently ".an down leads, looked for further refer- 
ences that were listea in things you read"; at another college it 
was 15%. These differences suggest the special role that may be 
played by certain prominent, and perhaps also attractive, physical 
facilities. In any case, since all six of these colleges described 
their missions in very similar terms, the fact that some aspects 
of student life are quite different at the campuses would enable 
each college to claim, and document, a certain uniqueness. This 
could be an asset in student recruitipent and student retention for 
the college; and in capturing the attention of students and 
parents who may be engaged in "comparison shopping" about where to 
go to college by giving them specific examples of what students do. 

A third i llustration. of the di fferefktial value of the quality 
of effort scales is the very special case of the Bible colleges. 
Daniel Brown, in his doctoral dissertation, selected a representa- 
tive group of 14 Bible col leges-'-representative as to denominational 
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affiliation, region of the country, etc. —and obtained three 
sources of data about them: (1) characteristics of their incoming 
freshmen froro the annual freshman surveys conducted by Alexander 
Astin, (2) characteristics of their environments by administering 
Pace's College and University Environment Scales, and (3) their 
responses to Pace's College Student Experiences questionnaire. 
Apparently i Bible colleges had never been studied empirically in 
these respects. In some ways they are one segment of America's 
system of higher education; but in other respects they have regarded 
themselves as *'a breed apart." So Daniel Brown's survey brings to 
light for the first time the similarities and differences between 
Bible colleges and other institutions of higher education. 

Bible colleges are not liberal arts colleges with a special 
emphasis on Bible and religion. Rather, their primary mission, is 
in the preparation of ministers and others for Christian service. 
In student academic selectivity thay are generally similar to 
' Libeval Arts, Type II, institutions. Their student bodies are 
very homogeneous, and the Bible college environments are also very 
homogeneous. The scholarly emphasis in their environments is a 
little above the average for Liberal Arts, Type II, colleges. 

The quality of effort mean scores for^ Bible college students 
are similar to the mean scores at other types of colleges—neither 
higher nor lower than the highest and lowest me^n scores at the 
other institutions--on most of the scales. The big exception to 

.1 

this Is the Science La|?oratory scale, the mean at Bible colleges 
being more than six points lower than at any other type of school. 
On two other scales the Bible colleges had lower mean scores than 



other schools, but the differences were much smaller--!. 2 points 
lower than any other on the Conversation Topics scale, and .8 
lower on the Experiences in Writing scale. In general, it is not 
the quality of students' effort thAt differentiates the Bible 
colleges from other sectors of higher education. Indeed, their 
similarity to liberal ai'ts colleges of moderate selectivity is a 
major finding in Daniel Brown's study. 

Nevertheless, the Bible colleges are clearly differentiated 
from other institutions in other respects that are measured by the 
College Student Experiences questionnaire. On all the environment 
ratings regarding relationships--among students, with faculty, 
administration, and style of operation--the Bible colleges emerge 
as very supportive institutions, more so by far in administration 
and operational style, and also in friendly relations among students 
and relations with faculty that are described as helpful and 
encouragingi The other major aspect of the environment which has 
strong emphasis at the Bible colleges is the vocational emphasis-- 
a result which obviously reflects their major purpose of training 
people for the ministry or similar Christian service. The Estimate 
of Gains siection of the questionnnaire also shows a much higher 
percent of students at Bible colleges than at any of the other 
types reporting substantial gain/progress in vocational training. 
Other differences in gains reflect the curriculum at the Bible 
colleges. Very few students report substantial gains related to 
science objectives; and also, with respect to "gaining a broad 
gene>al education," 47% of Bible collge students report substantial 
gain, compared with 63% at the Liberal Arts, Type II, colleges. 



BEST COPY AVAIUBIJ . 71 76 



In the objectives that refer to personal and social development, 
the percentage of Bible college students reporting substantial 
gain Is higher than at ^ny other institutional type--specif leal ly 
the goals of developing values and ethical standards, understanding 
oneself, understanding others, and the ability to function as a 
team member. 

The Pervasive Significance of Quality, of Effort 

Not only does quality of effort have a general predictive 
value, a special diagnostic value, and some differential values, 
as the results thus far presented have shown. It also has a very 
pervasive value. By this we mean that the r'ange or scope of high 
quality effort Is related to the range or scope of. high achievement. 
The more aspects of the co;ilege experience (use of facilities and 
opportunities) one participates In at an above average level of 
quality of effort, the more objectives (different goals of higher 
education) one makes above average progress toward their attainment. 
Breadth of Involvement and breadth of attainments go hand in hand. 

Of the 14 quality of effort scales In the questionnaire, 12 
are answered by everyone (not all students live In a campus resi- 
dence facility and so do not respond to the Residence scale, and 
not all students have had a. science laboratary course and so do 
not respond to the Science Laboratory scale). Of the 12 scales 
applicable to everyone, four are mainly concerned with academic/ 
intellectual activities (course learning, library, faculty, and 
writing), four are primarily personal and Interpersonal (personal 
experiences, student acqualntences , conversation topics, and 
conversation level), and four are prL-r-'rily centered around group 
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facilities and associations (student union, clubs and organiza- 
tions, aithletic and recreational facilities, and cultural facili- 
ties related to art, music, and theater). We devised a "breadth 
index" which is defined as the number of scales (different aspects 
of campus life) on which a student's score is above the median of 
some baseline group. This baseline could be the median at one's 
own institution, or the median of all student responses at all 
institutions. The baseline we have used consists of the responses 
from 11,084 students at 41 colleges and universities that obtained 
their replies from a good cross-section of the students. Scores 
on the breadth index could range from 0 to 12, and in fact do so. 
Some students invest above average quality of effort on all 12 of 
the topics and some students invest above average quality on none 
of the 12 topics. The distribution of breadth scores for the 
11,084 students at these 41 colleges and universities is shown in 
Table 10. A breadth score of 9 or higher was obtained by 24% of 
the students; and about the same proportion (27%) had a breadth 
score of 3 or lower. The other 49% had scores in the middle range 
of the distribution. 

Using a breadth score of 9 or higher (the upper fourth) as a 
definition of "high breadth," large differences between one college 
and another were revealed. For example, at one college only. 8% of 
the student body had a breadth score of 9 or higher; whereas at 
another college 55% of the students had a breadth score of 9 or 
higher. Clearly at some colleges the vigor and vitality of what 
students put into the college experience covers a much wider range 
of activities, is much more pervasive, than is true at other 
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Table 10 
DISTRIBUTION OF BREADTH SCORES 
(11,084 students at 41 colleges and universities) 



Breadth score 


Percentage of all scores 


\ 

12 V 


3% 




11 


5 




10 


7 


24% High Breadth 


$ 


9 




8 


10% 




7 


10 




6 


10 


49% Medium ^readth 


5 


10 




4 


9 




3 


9% 




2 


7 




1 


7 


27% Low Breadth 


0 


4 
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colleges. Indeed, the breadth index for a college may be a very 
good indicator of the quality of its undergraduate program, or, 
more explicitly, of the quality of undergraduate student experi- 
ence at the college. 

Evidence that the breadth score might be a very good index of 
the quality of undergraduate education on the campus is clearly 
suggested by the strong relationships between breadth scores and 
outcomes. 

In Table 11 the students are divided into the three breadth 
categories--high, medium, and low; and for each group of students 
the percentage attributing substantial gain from their college 
experience with respect to each of the 18 objectives is shown. In 
every instance the percentage reporting substantial gain is greatest 
among the high breadth group, and smallest among the low breadth 
group. With respect to some goals the percentage of high breadth 
students reporting substantial gains is more than twice as large 
as the percentage amoi g the low breadth students. So, not only do 
specific quality of effort scales have a clear relationship to 
certain specific gains, but quality of effort as a whole has a 
clear relationship to all gains. The more you put into the college 
experience, the more you get out of it. 

Breadth of high quality effort is also related to students' 
satisfaction with college. This is true at each major type of 
institution, as Table 12 shows. The differences are not as large 
as they were between breadth dnd gains, but they are nevertheless 
statistically significant. 
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- TABLE 11 
RELATIONS BETWEEN BREADTH OF EFFORT AND GAINS 

^—i — 



Percentage reporting substantial gains among students with: 



GAINS 


low 

breadth scores 


medium 
breadth scores 


High . 
breadth scores 


VOC 


45 


• 

50 


60 


BROAD KNOWL 


53 


70 


80 


ARTS 


17 


34 


52 


LIT 


17 


35 


58 


WRITE 


34 




/I 


PHILS, CULTS 


40 


63 


78 


VALUES, ETHICS 


49. 


71 


87 


UND SELF 


63 


81 


90 


UND OTHERS 


oO 


ol 


Q1 


TEAM 


35 


55 


70 


HEALTH 


27 


42 


56 


SPEC ADV ED 


51 , 


63 


74 


ANAL, LOGIC 


49 


66 


79 


SYNTH, RE LS " 


55 


76 


88 


INO LRN ^ 


63 


82 


91 


SCI/EXP 


31 


36 


42 


SCI/TECH 


30 


35 


44 


QUANT TH 


37 


45 


55 



•/ TABLE 12 

RELATIONS BETWEEN BREADTH OF EFFORT AND SATISFACTION WITH COLLEGE 



Among Students With 
High, Medium, and Low Breadth Scores 



Percentage Satisfied 
(scores 6,7,8) 



At Doctoral Universities 
High breadth scores 
Medium breadth scores 
Low breadth scores 

At Public Cofflprehenslves 
High breadth scores 
Medium breadth scores 
Low breadth scores 

At Private Comprehenslves 
High breadth scores 
Medium breadth scores 
Low breadth scores 

At Liberal Arts, Type I 
High breadth scores 
Medium breadth scores 
Low breadth scores 

At Liberal Arts, Type II 
High breadth ^cores 
Medium breadth scores 
Low breadtl^ scores 



83 
76 
68 

76 
70 
62 

74 
67 
56 

87 
81 
68 

76 
64 
66 
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Breadth of effort also differs by institutional type. The 
percentage of high breadth students at the selective liberal arts 
colleges is twice as great as it is at doctoral universities or at 
the public coinprehensives. These latter types of schools are all 
large schooTTiTWh^reas^ all liberal arts colleges are small schools. 
Table 13 shows these differences. It is apparently easier, or 

more likely, to be involved in many aspects of the college experi- 

• / 

ence at the relatively small colleges than at the large institu- 
tions where, pi^esumably, more opportunities ^are theoretically 
available. Perhaps it is easier to dritt, or be relatively unin- 
volved, at large schools; whereas at the smaller places pressures 
to become involved in campus activities may be stronger. 

In an earlier study, based on data from 24 colleges, breadth 
of effort. was found to correlate .80 with breadth of gains. 
Institutions having the highest breadth index^ for quality of 
effort also had the highest percentage of their students reporting 
gains that were "above average." Put another way, the most lively 
institutions (breadth of high quality student effort) were also 
the most productive institution^ (breadth of high level achievement). 

A somewhat related notion is that of "value added." It has 
often been assumed that those who start at or near the bottom on 
some criterion are likely to make the largest gains, and that 
therefore an investment in the low starting group is especially 
effective. Unfortunately perhaps, that is not so. Low starting 
groups can be defined in two ways-- low because of low scholastic 
aptitude as measured by the SAT-Verbal, or low as defined ty the 
gains reported at the end of the freshman year in college. As to 
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TABLE 13 

PERCENTAGE OF STUDENTS AT DIFFERENT TYPES OF INSTITUTIONS 
^ WITH HIGH, MEDIUM, AND LOW BREADTH SCORES 



Percentages 

Doctoral Public. Private Liberal Liberal 

Universities ComprehenHA^e Comprehensive Arts I Arts II 

High breadth . 17 15 19 35 25 

Medium breadth 50 45 50 49 ' 56 

Low breadth 33 40 21 16 25 
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selectivity, the rank order correlation of 1.4 schools between 
selectivity (Freshman SAT) and breadth of gains, was .42. In 
other words, the schools that had the "b^st" students to begin 
with, also showed the greatest breadth or scope of gains made by 
the students. More pertinent, perhaps. Is the relation between 
academic selectivity and academic gains— academic gain defined as 
the cluster of objectives we have. called general education and the 
cluster called Intellectual skills. Between SAT-Verbal and general 
education gains, at the 24 colleges, the rank order correlation 
was .57; and between SAT- Verbal and gains In Intellectual skills 
the correlation was .66. In other words, the more academic ability 
you have to begin with, the more you are likely to gain In academic, 
Intellectual achievement In college. 

A still more pertinent comparison Is between gains At the end 
of the freshman year and gains at the end of the senior year. 
These are, of course, differences between two cross-sections of 
students, not differences between the same students at a later 
period of time. But any college would surely hope that its seniors 
who are abouf^ to graduate would report gains on important educational 
objectives that were greater than the gains reported by Its end-of- 
year freshmen... Onft technical problem here is how to define the 
gains-- is It merely the difference in percentages between fresh'*ien 
and seniors who claim substantial gain, or is It the percentage of 
possible gain that should be used? If 60% of end-of-year freshmen 

**».' I 

say they have made substantial progress toward some goal, and 80% 
of seniors say they have made substantial progress, one could 
report the difference in two ways: first, that there Is a differ-. 
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ence of 20 percentage points between freshmen and seniors, or, 
second, that the gain is 50% of the possible gain (if one starts 
U 60%, the possible gain to 100% is 40; so if one ends at 80%, 
the gain is 50% of the possible gain). If one considers Just the 
raw difference ih percentages between freshmen and seniors, the 

rank order correlation at the 24 colleges between freshman and 

■ rf. 

senior status is approximately zero. But if one considers gain as 
the percentage of possible gain, the rank order correlation is 
.46. So, relative to possible gains, the more you know or have in 
the first place, the more you are apt to gain. 

Based on a much, larger 'sample, and using students as the unit 
of analysis, differences in gains reported by freshmen and by 
seniors are shown for each institutional type in Appendix Tables 12, 
13, 14, 15, and 16. Comparing these results with data previously 
reported on the quality of effort scores at the different types of 
institutions again illustrates the pervasive significance of 
quality of effort: for institutions with the highest gains toward 
tiie most goals also have the highest scores on most quality of 
effort scales, and vice versa. 

In some aspects of the college experience, quality of effort 
scores are not successively higher with each year in college. 
Perhaps the experience tends to occur early, reaching a peak by 
the end of the freshman or sophomore year. This is often the case" 
with the "non'academic" aspects of college-* such as use of the 
student union and athletic facilities, and also with respect to 
clubs and organizations and student acquaintances. One would not 
expect large additional gains when much effort and much progress 
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had already occurred during the freshman year, or during the first 
two years. In the "academic" aspects of colJege—such as use of 
the library, contacts with faculty members, course learning 
activities, science laboratory, and, to some extent, the informa- 
" tlonal level of student cohversations--one typically finds quality 
of effort scores that are higher each year from the freshman to 
the senior year, It is also progress toward objectives related to 
intellectual skills that show the greatest difference between the 
, responses of freshmen and seniors. 
Reading. Writing, and Computing ♦ 

The 1993 second edition of the College Student Experiences 
questionnaire contained new items about the extent of students' 
reading and writing activities, and about their experiences with 
computers. Both the first and second editions of the questionnaire, 
included the quality of effort scale labeled Experiences in Writing, 
and the objective described as "Writing clearly and effectively." 
Because there hdS been much concern about the reading and writing 
skills of college students, the increased enrollments in remedial 
English, plus the new interest in computers, we have taken a 
special look at what students report about these aspects of the 
college experience. 

Responses to the new items come only from the eight schools 
that used the second edition of the questionnaire in the Spring of 
1983. Although there were altogether 2299 students, half of them 
were from two docto^^al universities, and one should not, from this 
composite group of responses, draw any generalizations about the 
typical *»ctivities of today's college students. But one can 
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report the results for single institutions, unnamed, and note the 

very large differences that were found. 

Unlike ot^r items in the questionnaire, the new Item about 

writing activities asks for a report of what has been required by 

the college,, not a report of student initiative and effort. Two 

types of writing experiences were asked about, as follows: 

During the current school year, about how many written reports 
have you made? 

Essay Exams in Term Papers or Other 
Your Courses Written Reports 

. None 

Fewer than 5 

Between 5 and 10 

Between 10 and 20 

. More than 20 



At one institution 51% of the students reported that they had 
either none or fewer than five essay exams in their courses during 
the year. At anothar school the* corresponding percent was 22%. 
Also, at the first school 18% reported having had between 10 to 20 
or more essay exams during the year; and at the other college the 
figure was 38%. The college where students had the most essay 
exams also had the most term papers or other written reports, with 
49% of their students writing from 10 to 20 or more term papers or 
other written reports during the year. At another college only 
10% had that much writing required of them. There was one institution 
where 13% of the students had hO essay exams, and 14% had no term 
papers or other written work during the entire school year. 
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Practic© makes progress, or at least so it seems from the 
following comparison. At the college where students reported, 
having the most essay sxams and the most number. of required papers, 
79% of the students felt they had made substantial progress in 
their abil-it:/ to wrHe clearly and effectively; but at the college 
where students reported the fewest essay exams and the fewest 
required papers, 48% felt they had made substantial progress in 
their ability to Write well. 

A further analysis of writing activities, based on 13,000 
students from 45 institutions using the 1979 edition of the question 
naire, was made by Karen Lefever. Although, as previously reported 
in Appendix Table 9, differences in mean scores on the Experiences 
in Writing scale between Institutional types were very small, 
there were nevertheless several differences in certain specific 
activities. For example, at the selective liberal arts colleges, 
31% of the students said that they frequently spent at least five 
hours or more writing a paper (not counting time spent in reading 
or at the library); whereas at the other types of schools the 
corresponding percents ranged from 57% to 66%. Also, students at 
the selective liberal arts colleges were somewhat more likely to 
think about grammar and syntax while writing, and to write a rough 
draft and then revise it before handing it in. 

More important than these few differences, however, were the 
consistent patterns of relationship between writing and other 
variables, irrespective of institutional type. At each of the 
five institutional types, scores on the writing activity scale 
correlated .35 and above with the other academically oriented 
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scales (library, faculty, and course learning). Also correlations 
of .30 and above were found consistently^ between wr.iting activities 
and two of the personal/social scales (personal experiences and 
student acquaintances) and both of the conversation scales. 
Moreover, gains in writing ability were consistently related to 
other outcomes, irrespective of institutional type. Between 
progress in writing and gaining a broad general education, the 
correlations were typically about .30; between progress in writing 
and gains in various intellectual skills (analysis, synthesis, and 
independent learning), the correlations were typically about .35; 
and between writing progress and gains in knowledge and enjoyment 
of literature, the correlations were typically about .45. From 
these analyses one might speculate that writing is an activity 
that helps to integrate the students' educational experience and 
progress. 

The amount of reading students do--assigned and non-assigned— 
also varied greatly among institutions. The question about reading 
was as follows: 

During the current school year about how many books haN'e you 
read? y 

Textbooks or 

Assigned Books Non-Assigned Books 

None 

F wer than 5 

Between 5 and 10 

Between 10 and 20 

More than 20 
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At one institution 47% of the students reported that they had read 
at least 20 textbooks during the year. At this particular school 
about one-third of the students were humanities majors. At another 
school only 14% had read that many texts. A similar contrast 
existed for the reading of non-assigned books, with 10% at one 
school reading 20 or more, and 2% doing so at another school. At 
most colleges about 60% to 70% of the students said that they had 
read fewer than five non-assigned books. At colleges where the 
least amount of reading was required or done voluntarily, many of 
the students were majors in Business. One might suppose that the 
school where students read the most textbooks would compensate for 
this reading load By reading fewer non-assigned books, but that 
was not the case. Reading seems to be a general habit-- the more 
assigned books you read the more non-assigned books you also read. 

-. With regard to experiences with computers, most students, 
from half to three- fourths at the eight colleges, reported that 
they had never worked on a project using a computer, used a computer 
to assist in course learning, written a program to analyze data, 
or sought out-of-class instruction in using computers. For about 
one-third of the students at each of the colleges, "computers and 
other technologies" were never among their conversation topics; 
although about one-half occasionally talked about the topic. From 
these figures about activity it is not surprising to find similar 
figures about progress. At most of the colleges, from 40% to 60% 
of the students reported that they had made "very little" gain/ 
progress in "acquiring familiarity with the use of computers." 
Progress toward this objective is no doubt dependent on the 
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availability of equipment at the colleges, and the nature of 



assignments and prooably not attributable primarily to lack of 
interest or effort among the students. At one college, where many 
of the students were Business majors, and where computer equipment 
was surely available, nearly half of the student body claimed 
substantial progress in acquiring familiarity with the use ot 
computers. 

Institutional Research. Self-Study, and Evaluation 

Many of the analyses we have made of our inter- institutional 
data base can also be made within a single institution. The 
general questions simply become local questions. What are the 
best predictors of student gains and satisfaction at our school? 
What are the differences at our school between residents and 
commuters, or between transfer students and others? What are the 
differences between freshmen and seniors at our school? What is 
the breadth of high quality effort among our students? 

There are other analyses that may be more appropriate within 
institutions than between institutions. If one wished to compare 

t 

the experiences of Hispanic/ Mexican-American students with "majoritx 
students, one might 1n a national sample without suitable statistica 
controls simply be comparing students in southwestern colleges 
with those in other parts of the country. If one wished to compare 
"foreign" students with U.S. citizens, the results might be quite 
different at different colleges. At UCLA, for example, many 
foreign students are Oriental/Asian. At some other college the 
foreign student population may come mainly from Saudi Arabia. 
Also, in comparing the experiences of foreign students with U.S. 
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students one needs to take account of the students' major field 
and other factors which may influence the results. Sensitivity to 
such factors is mainly a local responsibility for the obvious 
reason that those who are on the scene are more likely tc knew 
about them. 

When the College Student Experiences questionnaire was first 
published in 1979, we obtained responses from a random sample of 
UCLA undergraduates and subsequently made a number of analyses 
which we believed would be of interest to the UCLA administration. 
Earlier, we had distributed a copy of the questionnaire to various 
administrators and then met with a group of them to ask whether 
there were any analyses or comparisons that would be particularly 
useful to them. The group, about a dozen people, included 
individuals from the chancellor's office, the planning office, the 
office of the vice chancellor for student affairs, deans of several 
divisions in the college of letters and science, and other academic 
personnel. From their discussions, and following their advice 
that a series of very brief and clearly focused reports would be 
more useful than a single comprehensive report, we prepared and 
delivered copies of such reports to the chancellor's office and 
the planning office where they could then be duplicated and 
distributed to selected individuals. Tha.e were eight topics about 
which special interest had been expressed'-course learning, use of 
the library, writing experiences, contacts with faculty, students 
who lived in the dormitories, transfer students, student satisfac- 
tion, and minorities. No analysis was made of the educational 
experiences of minority students because the number of cases was 
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too small to assure adequate reliability. In addition, we prepared 
three other reports which we regarded as 1mportant--a report on 
relationships between the data about effort, environment, and 
attainment; a report about breadth or scope of high quality effort; 
and a report about predicting students' gaihs/progress. 

In most of these reports roughly similar types of presentation 

f 

were made: first, the overall percentages of students who responded 
"never," "occasionally," "often," and "very often" to the activities 
in the particular scale; second, the correlations, or sometimes 
differences between high and low participants, between effort and 
such other variables as year in school, grades, aspirations for 

0 

further education, sex, race, transfer status, residence, etc. as 
appropriate to the topic; and third, relationships between effort 
and gains. Most of the reports were about three pages in length, 
and were self-contained in the sense that one did not need to read 
any additional reports in order to understand the results presented. 

In 1983 a new opportunity to survey UCLA undergraduates 
occured. The University was scheduled for its ten-year accredita- 
tion review cycle by the Western Association in 1984; and 1983 was 
to be the time for its self-study. I suggested to the chancellor's 
office that a survey of undergraduate student life migf>t be appro- 
priate. Research oriented universities such as UCLA have a great' 
deal of external and reasonably objective data about their eminence 
in research and scholarly productivity; and these external ratings 
put UCLA among the top three or four universities in the U.S. But 
what do we, or any other major university, really know about the 
quality or distinction of the undergrc(duate program? Ratings of 
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department chairmen at other universities, of the sort that are 
used in judging graduate program distinction, are of little value 
for this purpose because department chairmen know little or nothing 
about undergraduate programs at other universities. Moreover, the 
quality of the undergraduate experience is influenced by many 
elements that are unrelated to the departmental major--elements 
such as student associate. ..s, relations with faculty, clubs and 
organizations, etc. Institutions that may be eminent in certain 
respects may not be especially distinctive in others. Institutions 
that have an excellent undergraduate program may not nave any 
program at all in doctoral level research, and training. By taking 
the initiative to survey the quality of its students' undergraduate 
experience and education, UCLA could make a special contribution 
to the value of the accreditation review process, and at the same 
time obtain data that could stimulate lively and ultimastely produc- 
tive debate within the university, leading, one might hope, to 
more effective and more distinguished undergraduate programs. The 
Chancellor's off ice^agreed with the suggestion and provided the 
support necessary to conduct the survey. 

During April and May of 1983, 846 undergraduates responded to 
the College Student Experiences questionnaire. The number repre- 
sented a 46% return from a randomly selected sample, and reflected 
reasonably well what is known about UCLA undergraduates from other 
sources. Analysis of the results have been made by Karen Lefever. 
One initial interest was. to see whether the new sample of responses 
differed from the 1979 sample.. For the most part the results were 
very similar, but there were a few changes that were especially 
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pleasing. The 1983 students had significantly higher mean scores 
on the quality of effort scales regarding library use, contacts 
with faculty, and writing; and they also had significantly higher 
scores on the satisfaction index. In outcomes, the main difference 
was that a higher percentage now reported substantial gain in 
writing effectively, a result that may have been facilitated by the 
increased academic eimphasis ahd resources the university had put 
into writing programs. Several institutions have now used the 
questionnaire on more than one occasion in order to study change. 
When new programs or facilities have been introduced, the ques- 
tionnaire is apparently sensitive enough to reveal changes in 
students' experience that were intended by the college; and this 
is an important element in good evaluation. 

Another ojvious interest in the, self-survey\^as to see whether 
the UCLA student responses would in any important ways be different 
from other doctoral universities. Compared with a composite set 
of 3500 responses from underflT^atkiates at ten doctoral granting 
universities the responses from UCLA stud«»''t'= were not sharply 
different in most respects. For example, with respect to the 
environment ratings, UCLA was a little stronger in its emphasis on 
esthetic qualities, and on critical and analytical qualities. , 

r 

With respect to student relationships, the percent of UCLA students 
rating them as friendly and supportive was a little lower (63%) 
than the percent at other universities (68%). As to "scores on the 
various quality of effort scales, UCLA was significantly higher on 
the Writing scale and the Student Union, scale and significantly 
lower on the Faculty scale, the Art, Music, Theatre scale and the 
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Athletic and Recreational t'acilities scale. With respect to 
various outcomes (gains, progress), UCLA had a higher percent 
reporting substantial gains in writing, and lower percentages on 
the several objectives related to personal and social development 
and understanding. There were no differences between UCLA and the 
composite of doctoral universities with respect to the intellectual 
emphasis on the environment, the reported gains toward the various 
goals of intellectual competence, the quality of effort in library 
use, or in course learning. 

t 

In answer to the question "How well do you like college?" 
44% of the UCLA students said "I am enthusiastic about it," compared 
with 34% at other doctoral universities. To the question "If you 
could start over again, would you go do the same college you are. 
now attending?" 85% of UCLA students said yes, compared with 80% 
at other doctoral universities. 

Here are some other highlights that are reported in the self 
survey. 

The goals toward which the largest number of undergraduates 
believe they have made substantial progress are ones related to 
the development of intellectual competencies--analysis and logic, 
synthesis, capacity for independent learning, and the acquisition 
of specialized knowledge for advanced work. These same goals are 
also the ones which show the greatest gains, as inferred from the 
relative differences between the responses of end-of-year freshmen 
compared with the responses of end-of-year seniors. 

The goals toward which the next largest number of students 
report substantial progress are those related to self-understanding. 
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understanding others, the clarif icat 'on of personal values, and 
broad general education about different fields of knowledge. 

In both of tht above categories, the proportion of students 
indicating substar.tia.l gains are generally from two-thirds to more 
than four-fifths; and almost none of the seniors (from 2% to 5%) 
report having made "very little" progress in the above directions. 

Goals that related to science, on the one hand, and to litera- 
ture and arts, on the other hand, are apparently so embedded in 
the respective curricula that substantial progress toward science 
goals or toward literature and arts is made only by students who 
major in those fields. Among *he total sample of undergraduates 
roughly 1/3 to 2/5 reported substantial gains in sciences; and 
roughly 1/3 to 1/4 reported substantial gains in literature and 
arts. The contrasts are sharp. Among majors in sciences or 
engineering, 2/3 to 3/4 report substantial gains related to those 
fields, but generally about 1/10 to 1/5 report substantial gains 
related to literature and arts. Similarly, among majors in Arts 
and Humanities, about 2/3 to 4/5 report substantial gains related 
to those fields, but generally about 1/10 to 1/5 report substantial 
gains related to the sciences. 

In characterizing the UCLA environment, nearly everyone (85% 
to 90%) regards UCLA as having a "strong emphasis" on the develop- 
ment of a student's academic, scholarly, and intellectual qualities; 
but generally only 1/3 to 1/2 of the students describe UCLA as 
having a strong emphasis on the development of a student's vocational 
and occupational competence. Also, generally about 3/5 to 2/3 of 
the students consider the relationships among students as friendly 
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and supportive, and about the same proportions describe relation- 
ships with faculty as helpful and encouraging. 

Some of these comparisons prompted discussion about general 
education, especially because the university had recently modified 
its distribution requirements; yet science and non-science students 
remain quite far apart in many of their experiences and attainments. 

Other results prompted discussion of undergraduate teaching. 
For example, in some fields the proportion of students rating the 
faculty as '^approachable, helpful, understanding, and encouraging 
was typically 7/10 to 3/4, whereas in other major fields it was 
about 1/2. 

One of the merits in a self-study survey of this kind is in 
the questions it raises--the results are rarely pleasing to all 
observers. Specific data help to focus discussion, and raise such 
questions as: Why? What would happen if? How could this be 
modified? 

During the 1983-84 school year many specially focused analyses 
of the College Students Experiences results at UCLA will no doubt 
be made. Those who are particularly concerned with foreign students 
want to know about the experiences, progress, and satisfaction of 
foreign students, both immigrants and non-immigrants. Those who 
administer the residential life programs want to know more about 
the dormitory residents, about students who live in the fraternities 
and sororities, etc. Those who plan the orientation programs for 
freshmen want to know whether the freshmen experience has been a 
rewarding one for most students. Those who are particularly 
concerned with personal counseling need to know more about the 
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"uninvolved" students: these are the opposite of the "high breadth" 
students. Who are they? Are they also the dissatisfied students? 
Are they the future dropouts? Can their experience be enriched? 

This monograph is not the place to report clll the results of 
the UCLA survey. We mention its existence, and some of its findings, 
primarily to illustrate the potential value of the College Student . 
Experiences questionnaire in the self-study aspect of accreditation. 
The , accreditation associations want evidence about outcomes as 
well as about processes, programs, resources, administration, 
finance, etc. The questionnaire provides data about what students 
do with the resources and what they get out of them. As more 
colleges use the questionnaire, more data will be available for. 
normative comparisons. Additional thoughts about research and 
evaluation in higher education, and about the significance of 
results so far attained, are the subject of the final section of 
this monograph. 
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REFLECTIONS 

Quality of effort is a useful si measurable concept. The 
original thought that measuring students' quality of effort may 
be" the key to evaluating the quality of undergraduate education 
has been supported by many of the analyses over the past five 
years. Quality of effort is the best predictor of students' 
progress tovrard the attainment of important educational goals. 
Granted all the elements that account for the selective distri- 
bution of who goes where to college, once the students get there 
what counts most toward their attainment is not who they are or 
where they are but what they do. It is the quality of effort they 
put into capitalizing on the resou*^ces and opportunities for 
learning and development that exist in the college setting that 
makes the difference. Further/the breadth or scope of student 
effort is clearly related to the breadth of outcomes toward which 
students make substantial p)^ogress, as well as progress on each 
specific objective. The more -students put into their college 
experience the more they get out of it. 

Quality of effort, and the breadth of high quality effort, 
are indications of intitiative. Almost all of the activities in ^ 
the quality of effort scales are voluntary ones. Going to college 
is in itself a voluntary activity. In college, students don't 
have to browse in the library stacks, or make outlines from their 
class notes, or work on a committee, or have serious discussions 
with students whose religious beliefs are very different from 
their own. But these activities and all the others in the ques- 
tionnaire are ones that can and do contribute to learning and 
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development. They are activities that professors, counselors, and 
other educators regard as desirable; but they are not required 
activities. In higher education there is no detailed regulation 
or monitoring of how students spend their time. As education 
moves from kindergarten to college there is progressively less 
supervision and more individual freedom and responsibility. The 
developmental and educative process from elementary school to 
secondary school to college is not only one of acquiring more and 
more knowledge but also one requiring more and more initiaj^ive. 
In this respect one might say that college can't give you an 
education, but if you to go college and make the effort to use the 
facilities and opportunities it provides you can ^et a very good 
education. 

Despite the clear diV'eqtion of these results emphasizing the' 
importance of student initiative, one should not conclude that 
what the college does is of minor influence. It is the college 
that provides the facilitieji and resources in the^first place. It 
is the coTlege^-the administration as well as the professors--that 
sets the intellectual standards, the quality of performance it 
expects from students, and exemplifies its values by the quality 
of the facilities it provides. 

There is no doubt a connection between students' quality of 
effort and the quality of facilities and opportunities that make 
the effort worthwhile. The quality of effort of students is high 
in the elite liberal arts colleges that have very good facilities. 
Although the major research universities probably have the most to 
offer--the best libraries, the most distinguished research scientists 
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and scholars, the most v/ell -equipped facilities in the arts, 
music, and theater, tt^^e best athletic and recreational facilties, 
etc. --the typical studi^nt iq-'those universities does not gain as 
much as the typical student in the elite liberal arts colleges. I 
suggested earlier that the big schools it is easier to drift, 
partly because no one really knows that a student may be drifting; 
whereas in small schools one cannot hide from one's fellow students 
or from the faculty. The crucial importance of students' iriitiative, 
however, is document^ed by the fact that, in the big. doctoral 
universities with' their rich resources, the students who really 
make the effort (the -^op 30% on the "breadth index" for those 
schools) gain more than the typical student in the best liberal 
arts colleges. This simply suggest that the quality of outcomes 
depend not only on the students' quality of effort but alsja' on the 
quality of the resources and opportunities that are available. In 
the big schools, high level attainment is especially dependent on 
high level initiative. 

At the beginning of this report I suggested that there was a 
quality dimension to the educational process as well as to the 
educational product. If a college hopes to stimulate high level 
efforts by its students, then the scores of those students on the 
quality of effort scales are evidence of how well that objective 
has been achieved. In this sense, the students' quality of effort 
scores are a measure of the college's success; there is, in other 
words, a vital and stimulating campus life that permeates the 
institi't ion. This brings me to some further thoughts about the 
importance of "process." 
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The measurement of process is the missing link in past evalua- 
tions of higher education. We know a great deal about the char- 
acteristics of entering college students. The student question- 
naires accompanying the SAT and the ACT, as well as the freshmen 
surveys initiated by Alexander Astin at the ACE and continuing at 
UCLA, have given useful information about who goes to college and 
how the characteristics of entering college students have changed 
over the years. Moreover, surveys of freshmen have formed the 
foundation for follow-up studies with respect to such things as 
changes in occupational interests, changes in major fields, etc. 
although relatively few of, those follow-ups have extended beyond 
the years of college graduation. In any case, we have had, over 
the years, a moderately important baseline as to what people" were 
like when they entered college. 

We also have a fair amount of information about what students 

learn in college, based on forty or more years of achievement 
testing with such instruments as the Area tests of the Graduate 
Record Examinations, '&ther tests of general education, and the 
achievement tests in many major fields that have been constructed 
by the ETS. We have measured students* performance on a good many 
objective and well constructed achievement tests. These measures 
have not often been used on a before and after basis ^ but never- 
theless .when they have been, particularly the tests in one's major 
field, the results have unif6rm1y shown substantial gains in test 
scores. The one thing students do indeed acquire in college is 
new information about a lot of subjects. 
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We also know, from a number of studies over the years, what 
people are like after they leave college. We have inquired about 
their employment, income, and job satisfaction. We ha^ve also 

I 

inquired about their' interests and participation in civic and 
cultural affairs. In many alumni studies questions about attitudes, 
values, and opinions have also been included, especially opinions 
about their college experience and the values they attribute to 
it. 

With respect to the process of education--what occurs between 
eritrance and exit--most of the dimensions in past research have 
been more or less static or unsystematic: for example, whether 
students lived on campus, what their major field was, what courses 
they took, whether they engaged in various extracurricular activities, 
etc. Some evaluations have focused on what the teachers do. 
These inquiries have, of course, been valuable and informative. 
They have not, however, dealt directly or comprehensively with 
what I believe is the most significant process dimension - the 
behavior of students. The Col lege Student Experiences question- 
naire is a systematic, conceptually based, comprehensive inventory 
of how college students spend their time in using the facilities 
and opportunities for learning and development that the college 
makes possible. 

At some future; point, just as we now have systematic inventories 
of students on arrival, and achievement test measures of their 
subject-matter learning, and periodic surveys of what they are^ 
like some years after graduation, we need to introduce on an 
equally large-scale, systematic basis what has been, until now, 
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the most significant missing element in understanding the process 
and effectiveness of higher education-^namely , what students 
themselves do when they are in college. ^ 

It would be incorrect to say that researchers have not been 
interested ii> students* behavior. Teachers, administrators, 
counselors, and many researchers have always been interested in 
and concerned about students behavior--they observe it, deal with 
it, stimulate It, reward it, day in and day out. There are also 
many measures of students' behavior if behavior is broadly defined 
to include attitudes, interests, values, and personality traits. 

It is, however, the. pragmatic scope of the activities in the 
College Student Experiences questionnaire that gives it a special 
value. Many of the activities reveal how students use faciliites 
on which, the college spends a lot of mon^y. Such information is 
particularly useful to those who manage the facilities; and the 
specificity of the activities suggests exactly where to focus 
attempts to stimulate better usage. So too, many of the activities 
involving interpersonal associations have clear rele^vance to the 
concerns of student personnel administrators. For example, if 80% 
of the students report that they have never talJ^ed with a counselor 
or other specialist about problems of a personal nature, it is 
because they don't really have any problems? 

The fact that the questionnaire is organized around campus ■ 
facilities and opportunities and in relation to educational goals 
distinguishes this instrument from other student questionnaires 
which are usually organized around* psychological dimensions. 
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Quality of effort is not the same as motivation. Motivation is 
often described as a general psychological phenomenon. Quality of 
effort is more specifically an educational activity related^to 
specific aspects of educational experience, ^Nor is quality of 
effort the same as persistence, far persistence does not include 
initiative. Nor is quality of effort a_ personal ity trait, for 
scientists and engineers exhibiting high quality of effort in 
various aspects of their educational experience are quite different 
in personality from humanists and artists who are also exhibiting 
high quality of effort. The essence of quality of effort, if it 
has an essence, may be identified in future research. Meanwhile, 
we know that it is a practical and powerful concept that can 
enrich our understanding of student learning and development in 
college. It is a variable that future research can ill afford to 
ignore. 

There is another aspect of the College Student Experiences, 
questionnaire that is very timely in view of the belief that the 
effectiveness of college education should be judged on a "value 
added" basis. My own opinion is that "value added" is an interesting 
idea, but an idea not yet well defined. It js clearly inappropriate 
if one uses achievement other tests on which the brightest and 
best informed students make such^high scores initially that the 
test is incapable of identifying added value. It is redundant 
when before and after differences are obvious. Nevertheless, 
(although it may have been .fortuitous rather than foresighted on my 
part, thp question in the Estimate of Gains section of the ques- • 



I 



tionnaire is a value added question. It doesn't ask students t^o 



ERIC 



1^. BEST COPY AVAILABLfc 102 ^ 



estimate how much they have benefited from college, or how well 
they can do certain things, or how much they know. It asks how 
mJch they have gained, how much they tiave added to. their knowledge, 
ttieir intellectual skills, and to other abilities and insights as 
9, result pf their experiences in college. 

What they were like at entrance is self defined, and what 
progress they claim is also self defined; but the response is a 
value added judgment. When students fill out the questionnaire 
they are engaging in recall,' introspection, and judgment. We know 
from both internal and external evidence that their recall of 
activities and their estimates of gain are credible, and that they 
respond carefully and perhaps in many cases with personal interest 
to the content of the questionnaire. Because their responses are 
congruent with other judgments, and because for some goals the 
students may well be the only qualified judges of whether they are 
any different today from what they were when they arrived, we must 
pay attention to what they say. 

Some distinguished educators, Clark Kerr among them, have 
said that in most respects higher education is a very successful 



institution, but that general education, or liberal education, is 
a disaster area. If this means that one cannot find a common 
curriculum, or any common core of knowledge, in the education of 
today's students, the judgment has some validity. But if one 
means that the goals traditionally associated with liberal education 
are no longer being achieved, the judgment has much less validity. 
Our evidence shows that most students (3/4 and more) have made 
substantial progress in sharpening their intellectual skills--the 
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ability to analyze and be logical, the ability to see relationships 
and to synthesize, the ability to pursue ideas and find information 
one needs. Our evidence also shows that most students (3/4 and 
more) have made substantial progress in understanding themselves 
and others, and in clarifying their values and ethical standards. 
Our evidence further si.jws that most students (2/3 or so) make 
substantial progress toward gaining a broad general education 
about different fields of knowledge, and becoming aware if different 
philosophies, cultures, and ways of life. The approximate percentages 
(they vary somewhat at different types of institutions) are percentages 
claiming substantial gains ("quite a bit" or "very much" progress). 
If one adds those who claim at least "some" progress, rather than 
"very little," all of the above percentages approach 100%, usually 
coming to 90% and more. 

In the Appendix Tables 12 to 16 where percentages of freshmen 
and seniors reporting substantial gain are shown for each institu- 
tional type, one needs to add a cautionary note. There are a few 
instances in which the reported gains of seniors are less than the 
reported gains of end-of-year freshmen. In many institutions 
there are large numbers of transfer students, and therefore all 
the seniors are not ones who have spent four years or more at the 
school. The difference in percents needs to be interpreted in 
this light. At the public comprehensive universities in our 
samples, 45% of the respondents were transfer students. In the 
doctoral universities, the private comprehensive universities, and 
the liberal arts II colleges, about 1/5 were transfer students. 
In liberal arts I colleges about 1/8 were transfer students. 
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In a very real sense, the entire experience of higher education 
is a value added experience. Every event that occurs in college 
and that is meant to contribute to student learning and development 
is a value added event. One could think of value added units as 
all. things that did not exist before and so have been added to 
one's life--courses taken, exams passed, papers finished, concerts 
attended, etc. , etc. , etc. 

What our research shows is that this value added element, or 
in our terms the percent of students who make substantial progress, 
toward the attainment of important goals of higher education, is 
primarily the result of the quality of effort students put into 
their education. 
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INFORMATION ABOUT OBTAINING AND USING THE QUESTIONNAIRE 

Normally, colleges give the questionnaire to samples of their 
undergraduates in the, Spring of the year. 

When the questionnaires have been filled out by the students, 

» 

the college sends the completed questionnaires to Intran Corp. , 
4555 W. 77th Street, Minneapolis, MN 55435. Intran "processes" 
the results. The college gets a tape or cards containing all the 
responses^ of all its students and all the scores on the various, 
scales. The college can then analyze its results in whatever says 
it wishes. 

The college also gets, from the Higher Education Rev.earch 
Institute, a selective print-out of its own results, and a composite 
report showing results from different types of colleges and univer- 
sities that have been accumulated over a period of several years. 

All orders i-biUing, payroei^ts, c^^^^^ should be 

addressed to the Higher Education Research Institute, UCLA Graduate 
School of Education, 405 Hilgard Avenue, Los Angeles, CA 90024. 
Costs are as follows: 

° Copies of the questionnaire at 40$ each 

° Processing completed questionnaires at $1.00 each 

° Basic institutional participation fee of $175.00 
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KEY TO ABBREVIATIONS 

QE = Quality of Effort 
The QE scales dre: 
LIB = Library Experiences 
FAC = Experiences with Faculty 
COURSE = Course Learning 
AMT = Art, Music, Theatre 
UNION = Student Union 
ATHL = Athletic and Recreation Facilities 
CLUBS = Clubs and Organizations 
WRITE = Experience in Writing 
PERS EXP = Personal Experience 
ST ACQ = Student Acquaintances 
SCI LAB = Science Lab Activities 
SCI/TECH = Science/Technology 
DORM, F/S = Dormitory or Fraternity/Sorority 
CONV TPS = Conversation Topics 
CONV INFO = Information in Conversations 
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KEY TO ABBREVIATIONS 

ENV = Environment 

The ENV ratings are: ' 

SCHOL = Academic, Scholarly, Intellectual Emphasis 
ESTH = Esthetic, Expressive, Creative Emphasis 
CRIT = Critical, Evaluative, Analytical Emphasis 
VOC - Vocational Emphasis 
REL = Personal Relevance of Courses 

STUDENT = Relationships Among Students--Friendly, Supportive 
FACULTY = Relationship with Faculty-Helpful, Encouraging 
ADM = Relationship with Administration--Considerate, Flexible 
OPER = Operational Style--Open, Adaptive 
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KEY TO ABBREVIATIONS 



GAIN = Gains/Progress 

The GAIN topics are: 

JOB = Knowledge for specific job or type of work 

SPEC ADV ED = Specialization for further education 

.1 ■ '■ • 

' BROAD KNOWL = General education about different fields of knowledge 
CAREER =: Information relevant to a career 
ARTS = Appreciation and enjoyment of art, music, drama 
LIT = Acquaintance and enjoyment of literature 
WRITE = Writing clearly and effectively 
COMPUTERS— Familiarity with use of computers 
PHILS, CULTS = Awareness of different philosophies and cultures 
VALUES, ETHICS - Developing values and ethical standards 
UND SELF = Understanding self--abilities, interests, personality. 

» 

UND OTHERS = Understanding other people and ability to get along 

TEAM = Ability to function as a team member 

HEALTH = Good health habits and physical fitness 

SCI/EXP = Understanding nature of science and experimentation 

SCI/TECH = Understanding new scientific and technical developments 

CONSQ SCI/TECH = Aware of consequences of new application in 
science and technology 

ANAL, LOGIC = Ability to think analytically and logically 

QUANT TH = Quantitative thinking 

SYNTH, RELS = Ability to put ideas together, see relationships, 
similarities 

IND LRN = Ability to learn on your own, pursue ideas, and find 
information you need 



•I 

^ BEST COPY AVAILABL, 

ERIC 



\ 

\ 



APPENDIX TABLE 1 

FACTdR ANALYSIS OF ESTIMATES OF (JaINS 
1979 DATA 
(N = 3006) 





Factor 1 




Factor 2 


Factor 3 


Factor 4 


Personal/Social 
Gains Topics Development 


General Education 
Literature & Arts 


Intel lectudi 
Skills 


Understandl no 
Science 


UNO OTHERS 


72* 




22 


17 


02 


UND SELF 


68* 




27 


21 


04 


TEAM 


64* . 




06 


14 


08 


VALUES, ETHICS 


59* 




38 


18 


.03 


HEALTH 


55* 




06 


14 


08 


LIT 


03 




79* 


05 


-01 


WRITE 


09 




53* 


28 


01 


PHILS, CULTS 


29 




Do 




no 


ARTS 


16 




52* 


-05 


-02 


BROAD KNOWL 


11 






1 a 




SYWTHv RELS. 


23 




-, -.--.3 a--- 


71* 


12 


ANAL, LOGIC 


17 




19 


67* 


36 


IND LRN 


35 




32 


47* 




SPEC ADV ED 


14 




13 


38* 


30 


SCI/EXP 


10 




02 


19 


86* 


SCI/TECH 


08 




05 


24 


83* 


QUANT TH 


14 




-06 


54 


47* 


VOC 


23 




, -02 


26 


15 


* Factor loadings 


--principal 


components with Varimax rotation 




* Decimal points 


are omitted 


in 


Tables L- 7 







ERIC 



BEST COPY AVAILAb: 



111 



116 



APPENDIX TABLE 2\ 



MAJOR PREDICTORS OF STUDENT GAINS/PROGRESS TOWARD 
THE ATTAINMENT OF IMPORTANT COLLEGE OBJECTIVES 
(PREDICTOR VARIABLES LISTED IN ORDER OF THEIR IMPORTANCE) 

1979 DATA \ 
(N = 3006) 



Multiple R 



Cljange .in R^ 



Personal/Social Development 

QE:Pers Exp 
QR:Athl 
ENV: Student 
Satisfaction 
QE:Conv Tps 
Years in college 
ENV:Voc 
QE:St Acq 
Other variables 

Total 



38 
48 
54 
56 
57 
58 
59 
60 
62 

62 



14 
23 
.29 
31 
33 
34 
35 
36 
3^ 

39, 



14 
09 
06 
02 

,02 
1 
1 
01 

0? 



Intellectual Skills 

QE: Course 
Satisfaction 
Year in college 
ENVrCrit 
QE:Conv Info 
QE:Sci Lab 
ENV: Faculty 
Sex (male) 
QE:Lib 
Grades 

Other variables 



36 
45 
49 
54 
56 
58 
59 
60 
61 
61 
63 



13 
20 
24 
29 
32 
34 
35 
36 
37 
38 
40 



13 
07 
05 
04 
03 
02 
02 
01 
01 
01 
02 



Total 



63 



40 



BEST COPY AVAILAP' 



112 il7 



m 



APPENDIX TABLE 2 (continued) 



Multiple R Change in 



General Education. Literature & Arts 

QE:AMT 43 18 - 18 

QE:Write 50. 25 07 

ENV:Crit 55 30 05 

ENV: Faculty 57 33 03 

QE:Conv Tps 59 35 02 

Major in Fine Arts/Hum. 61 37 02 

ENV:Esth 62 38 01 

Satisfaction 63 ^ 39 01 

QE:Lib 63 40 01 

Other variables 65 42 02 

Total 65 42 

Understanding Science 

QE:Sci Lab 52 27 27 

Major in Sciences 56 31 04 

Sex (male) 59 :34 03 

QE: Course .61 37^^ 03 

(neg) Major in Fine Arts/Hum 63 40 03 

Satisfaction 64 41 01 

Year in college 64 42 01 

Major in Engineering 65 43 01 

QE:Conv Info 66 44 01 

ENVrCrit 66 44 ~ " Or 

(neg) Lived on campus 66 45 01 

Other variables , 68 46 01 

Total 68 46 
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APPENDIX TABLE 3 



ADDED CONTRIBUTION OF QUALITY OF EFFORT 
TO THE PREDICTION OF STUDENT GAINS/PROGRESS TOWARD 
THE ATTAINMENT OF IMPORTANT COLLEGE OBJECTIVES 

1979 DATA 
(N = 3006) 



Multiple R R^ Change in R^ 



Personal /Social Development 

Student background 36 13 13 

College status 47 22 09 

Environment .49 24 02 

Quality of Effort 6^ 39 ,15 

Intellectual Skills 

Student background 10 01 - : oi 

College status 53 28 27 

Environment 55 30 02 

Quality of Effort 63 '40 10 

General Education, Literature & Arts 

Student, background 14 ' 02 02 

College! status 48 23 21 

Environment _ 55. 30 07 

Quality of Effort 66 43 13 

Understanding Science 

Student background 23 05 05 

Col log?? status 58 34 29 

Environment 60 36 02 

Quality of Effort 68 47 11 
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APPENDIX TABLE 4 



FACTOR ANALYSIS OF ESTIMATES OF GAINS 
^ SECOND EDITION, 1983 
(N = 2299) 



Factor 1 
Personal/Social 
Gains Topics Development 


/Factor 2 
Science/ 
Technology 


Factor 3 
General Education, 
Li terature & Arts 


Factor 4 
Intel lectual 
Skills 


Factor 5 
Vocation 


UND OTHERS 


-73* 


04 


18 


14 


11 


"UND SELF 


70* 


04 


25 


20 


09 


VALUES. ETHICS 


63* 


06 


31 


15 


05 


TEAM 


60* 


09 


05 


07 


21 


HEALTH 


50* 


17 


09 


05 


12 


SCI/TECH 


07 


93* 


00 


13 


13 


SCI/EXP 


14 


83* 


01 


13 


05 


CONSQ SCI /TECH 


12 


74* 


10 


17 


13 


COMPUTERS 


-03 


18* 


-10 


19 


19 


LIT 


08 


-02 


83* 


01 


-01 


WRITE 


12 


-03 


53* 


23 


02 


ARTS 


- 18 




-— - - -52*- - 


-09 


02 


PHILS, CULTS 


34, 


09 


y 46* 


14 


03 


BROAD KNOWL 


16 


07 


>■ " 43* 


23 


09 


ANAL, LOGIC 


21 


35 


20 


66* 


14 


SYNTH, RELS 


31 


14 


31 


64* 


09 


QUANT TH 


14 


44 


-05 


55* 


15 


INO LRN 


38 


07 


31 


43* 


19 


JOB 


16 


-04 


-09 


01 


73* 


CAREER 


25 


08 


15 


12 


61* 


SPEC ADV ED 


12 


26 


15 


21 


52* 
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APPENDIX TABLE 5 

MAJOR PREDICTORS OF STUDENT GAINS/PROGRESS TOWARD 
THE ATTAINMENT OF IMPORTANT COLLEGE OBJECTIVES 
(PREDICTOR VARIABLES LISTED IN ORDER OF THEIR IMPORTANCE) 

(SECOND EDITION, 1983) 
(N = 2299) 



• 


Multiple R ' 


n2 
R 


■ n2 

Change in R 


Perrsonal /Social Development 






• 

I 


V(C • uorni t / d 




ID 


1 a 

10 


unv • i\c 1 cVancc 




Li 


u/ 


Ocii 1 U 1 




CO 




l^C • nun 1 ^ 




51 
•91 




y(L • rcib LXp 


DV 




f\A 
UH 








VI £i 


QE: Course 


62 




m 

ux 


Major in health fields 


63 


40 




ENViCrit 


64 ' 


H i. 




(neg) Freshman 


64 


41 


01 

VIX 


Other variables 


68 


46 


05 


Total 


68 


*fD 




General Education, Literature 


& Arts 






QE;AMT 


49 


- - 24 


- . 24 


ENViCrit 


57 


32 


08 


Read texts 


61 


37 


05 


ENViEsth 


64 


41 


04 


ENV: Faculty 


65 


42 


01 


Major in Humanities 


67 


45 


03 


Senior 


68 


46 


01 


QE:St Acq 


69 


48 


. 02 


Major in Social Sciences 70 


49 


01 


(neg) Major in Engineering 


70 


49 


01 


Write papers 


71 


50 


01 


(neg) Freshman . 


71 


50 


01 


Other variables 


74 


55 


05 


Total 


74 


55 
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APPENDIX TABLE 5 (continued) 



Multiple g Change in 



Intel lectUtO Skills 

ENViCrit 43 19 19 

QE:Sci/Tech 52 27 08 

QE:Conv Info 57 32 05 

(neg) Freshman 60 36 04 

ENV: Faculty 61 37 01 

QE: Course 62 38 02 

Satisfaction 63 40 02 

(neg) Sophomore 64 ' 41 01 

Other variables 68 46 05 

Total 68 46 

Science/Technology 

QE:Sci/Tech 71 ,,50 50 

ENV: Relevant 72 S?^ 02 

Senior 73 ^^,^^3 01 

QE:Athl 73 53 • 01 

(neg) Major in Business 74 55 02 

QE: Write 74 55 01 

ENVtCrit 75 . 56 01 

Major in health fields 75 56 01 

Other variables 77 59 03 

Total 77 59 

Vocation 

ENVtVoc 43 '18 18 

(neg) Freshman 51 26 " 08 

ENV: Relevant 55 30 04 

QE: Course 58 34 04 

(neg) Major in Social Science 60 36 02 

(neg) Major in Humanities 62 38 02 

QF:Clubs 63 40 02 

Major in health fields 64 41 01 

Major in Education 65 42 01 

Senior 65 42 01 

ENV: Students 66 . 44 02 

QE:Athl 66 44 0.1 

(neg) Plan adv. degree 67 45 01 

Major in Computer Sci 67 45 • 01 

QETfacuHy. 67 45 01 

Other variables 70 49 04 

Total 70 49 
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APPENDIX TABLE 6 



ADDED CONTRIBUTION OF QUALITY OF EFFORT 
TO THE PREDICTION OF STUDENT GAINS/PROGRESS TOWARD 
THE ATTAINMENT OF IMPORTANT COLLEGE OBJECTIVES 
(SECOND EDITION, 1983) 
(N = 2299) 



Multiple R R*^ Change In R^ 



Personal /Social Development 

Student background 22 05 05 

College status 49 24 19 

Fnvironment 58 34 J"? 

Quality of effort 68 46 12 

Intellectual Skills 

Student background 12 01 01 



College status 



51 26 25 



Environment 61 37 11 

Quality of. effort 68 46 . 09 

General Education. Literature & Arts 

Student backc^ound 20 04 04 



College status 



61 37 33 



Environment .69 48 11 

Quality of effort 74 55 0/ 

Understanding Science 

Student background 15 02 . 02 



College status 
Envi ronment 
Quality of effort 



54 29 . 27 

58 34 . 05 

77 59 25 



Vocation 



Student background 17 03 03 

College status " It U 

Environment 67 45 

Quality of effort 71 50 05 
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APPENDIX TABLE 7 



MAJOR PREDICTORS OF STUDENTS SATISFACTION WITH 
COLLEGE IN FIVE TYPES OF INSTITUTIONS 
(N = 12,000, 1979 edition) 



Multiple R 



Change in R 



Doctoral Granting Universities . 

ENV: Student 31 

GAIN: Spec Adv Ed 38 

ENV: Faculty 40 

GAIN: Und Self 43 

ENV:Schol 45 

(neg) Grade mostly C or below 47 

QE:Dorm F/S 48 

Grade mostly A-, B+ 48 

' GAIN: Broad Knowl 49 

ENV:Oper 49 

QE:Athl 50 

Total all variables 55 

Public Comprehensives 

ENV:Oper 38 

ENV:Schol 46 

QE: Clubs 50 

(neg) Major field undecided 53 

QE:St Acq 55 

(neg) Major in sciences 57 

(neg) Sophomore 58 

GAIN: Spec Adv Ed 59 

GAIN: Values, Ethics 61 

ENV:Voc 62 

Major in Fine Arts 63 

QE:Dorm, F/S 64 

GAIN: Arts 65 

(neg) Grades mostly B-, C+ 65 

GAIN: Broad Knowl 66 

(neg) Junior 67 

QE:Athl 67 

Total all variables 75 



10 
14 
16 
18 
20 
22 
23 
23 
24 
24 
25 
30 



14 
21 
25 
28 
30 
32 
33 
35 
37 
38 
40 
41 
42 
43 
44 
45 
46 
56 



10 
04 
02 
02 
02 
02 
01 
01 
01 
01 
01 



14 
07 
04 
03 
02 
02 
01 
02 
02 
01 
02 
01 
01 
01 
01 
01 
01 
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APPENDIX TABLE 7 (continued) 



Multiple R Change in R^ 



Private Comprehensives 

ENV: Student 43 18 18 

ENV:Oper 49 . ' 24 06 

ENV: School 52 27 .03 

GAIN: Job 53 28 01 

GAIN: Lit 54 29 .01 

(neg) Grades mostly B-, C+ 55 30 01 

Total all variables 61 37' 

Liberal Arts. Type I 

ENV: Student 41 17 17 

ENVtCrit 48 23 06 

GAIN: Spec Adv Ed 52 27 04 

GAIN: Broad Knowl 55 30 03 

ENV: Faculty 56 32 02 

QE:Dorni F/S 58 33 01 

GAIN: Arts 58 34 01 

(neg) Grades mostly B-, C+ 59 35 01 

ENV:Oper 60 36 01 

Total all variables 64 41 

Liberal Arts. Type II 

ENV: Faculty 44 20 20 

ENV:Oper 50 25 05 

GAIN:Und Self 53 28 03 

GAIN: Broad Knowl 55 31 03 

ENV: Student 57 32 01 

Grades mostly A 58 33 01 

QE:Conv Tps 58 34 01 

ENV:Esth 61 37 02 

(neg) Grades mostly B-, C+ 62 38 01 

GAIN: Spec Adv Ed 62 39 01 

QE:Lib 63 39 .01 

Total all variables 66 44 
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APPENDS TABLE 8 
CORRELATES OF STUDENT SATISFACTION WITH COLLEGE 



At Doctoral 
Universities 




At Publ ic 
Comprehensives 




At Private 
Cojnprehensives 


At Liberal 
Arts, I 




At Liberal 
Arts, II 




ENV: Student 


.31 


ENV:Oper 


.38 


ENV: Student .43 


ENV: Student 


.41 


ENV: Faculty 


.44 


ENV: Faculty 


.27 


£NV:Schol 


.34 


ENV:Oper . .35 


ENV: Faculty 


.35 


ENV: Student 


.40 


ENV:Oper 


.22 






' ENVrSchol .25 


ENV:Crit 


.28 


ENV:Oper 


.37 


ENV:Schol 


.20 








ENV:Oper 


.26 


ENV:Esth 


.29 


GAIN:Sp Adv Ed 


.23 


GAIN:Sp Adv Ed 


.24 




GAIN:Sp Adv Ed 


.29 


GAIN:Und Self 


.32 


GAIN:Und Self 


.23 






GAIN: Broad Knowl 


.28 


GAIN: Broad Knowl 


.27 


GAIN: Broad Knowl 


.22 












GAIN;Sp Adv E(.^ 


.25 


QE: Dorm, F/S 


.23 


QE:Clubs 


.23 




QE:Dorm, F/S 


.28 


QE: Union 


.27 




QE:Dorin, F/S 


.23 
















QE:St Acq 


.22 
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APPENDIX TABLE 9 



INSTITUTIONAL DIFFERENCES IN QUALITY OF EFFORT MEAN SCORES 

(N = 14,615 at 62 schools) 



Mean Scores at Types of Institutions 



Quality of 
Effort Scales 



Doctoral 
Universities 



Public Private Libera'^ Arts 

Comprenensive Comprehensive Type I 



Academic. Intellectual Effort 



LIB 
FAC 

COURSE 
WRITE 
SCI LAB 



18.7 
18.7 
29.5 
24.2 
24.1 



Personal. Social Effort 

CONV TPS 25.9 

CONV INFO 14.7 

PERS EXP 21.3 

ST ACQ 24.2 

AMT • 19,8 

Group Facilities Effort 



DORM, F/S 
CLUBS 
UNION 
ATHL 



25.8 
19.2 
19.6 
18.4 



19.7 
19.4 
29.3 
24.2 
22.3 



24.9 
14.5 
21.0 
23.2 
19.1 



22.1 
17.7 
19.5 
16.1 



20.3 
20.8 
29.4 
25.0 
21.6 



25.8 
14.6 
22.4 
25.2 
20.7 



25.5 
21.6 
22.3 
18.9 



21.0 
21.9 
29.9 
25.1 
24.9 



26.9 
15.1 
22.3 
26.5 
22.8 



25.6 
23.6 
22.3 
20.4 



Liberal Arts 
Type II 



20.2 
21.5 
29.2 
25.0 
21.4 



25.4 
14.6 
22.6 
25.3 
19.8 

25.6 
21.5 
22.6 
18.8 
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APPENDIX TABLE 10 - 
INSTITUTIONAL DIFFERENCES IN ENVIRONMENT RATINGS 



Percentage with Ratings of 5, 6 or 7 (positive) on the Scale 



Environment 


Doctoral 


Public 


Private 


Liberal Arts 


Liberal Arts 


Universities 


Comprehensive 


Comprehensive 


Type I 


Type II 


Emphasis 












SCHOL 


84 


.70 


83 


95 


82 


ESTH 


43 


51 


55 


60 


59 


CRIT 


71 


.57 


67 


86 


62 


VOC 


50 


64 


58 


36 


66 


Relationship 












STUDENT 


68 


68 


76 


75 


80 


FACULTY 


62 


67 


78 


86 


82 


ADM 


34 


36 


51 


53 


• 61 


OPER 


38 


46 


48 


56 


52 
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APPENDIX TABLE 11 
INSTITUTIONAL DIFFERENCES IN ESTIMATES OF GAINS 



Percentage Reporting Substantial Gains 



Gains 



' Doctoral 
Universities 



VOC 



Intellectual Skills 

SPEC ADV ED 
ANAL, LOGIC 
SYNTH, RELS 
IND LRN 

Science 

SCI/EXP 
SCI/TECH 
QUANT TH 

General Education 

BROAD KNOWL 
ARTS 

LIT . 

WRITE 

PHILS, CULTS 



47 



64 
66 
71 
76 



44 
44 
48 



65 
27 
30 
46 
60 



56 



59 
61 
69 
76 



36 
39 
47 



63 
30 
27 
50 
55 



55 



62 
65 
75 
79 



30 
30 
46 



69 

3,6 
37 
5'5 
5:7 



36 



71 
.71 
79 
81 



41 

40 
43 



81 
.42 
39 
57 
72 



Public Private Liberal Arts Liberal Arts 

Comprehensive Comprehensive Type I Type II 



62 



61 
63 
73 
81 



29 
28 
44 



63 
30 
35 
58 
60 



130 



BEST COPY AVAILABLE 



ERIC 



APPENDIX TABLE 11 (continued) 



Gains 



zi(. 



Doctoral 
Universities 



1 .Perjcentage Reporting Substantial Gains 

Public Private Liberal Arts 

Comprehensive Comprehensive Type I 



Liberal Arts 
Type II 



Personal. Social Development 



VALUES, ETHICS 
UND SELF 
UND OTHERS 
TEAM 
HEALTH 



67\ 
76 \ 
76 \ 
49 ' 
38 



64 
75 
73 
55 
43 



69 
78 
78 

§5 
43 



75 
80 
79 
45 
34 



72 
80 
80 
60 
44 
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APPENDIX TABLE 12 



PERCENTAGE OF FRESHMEN AND SENIORS REPORTING 
SUBSTANTIAL GAINS AT DOCTORAL UNIVERSITIES 

(N = 2376) 



GAINS 



Percentage Substantial Gain 
Freshmen Seniors 



Change 
in Percent 



Change as Percentage 
of Possible Change 



Intellectual Skills 

ANAL, LOGIC * 53 

SYNTH. REL. 63 

INO LRN 69 

SPEC ADV ED 45 



73 
80 
85 
73 



20 
17 
16 
28 



A3 
46 
52 

51 



General Education 

BROAD KNOWL 63 

ARTS 23 

LIT 28 

WRITE 45 

PHILS. CULTS 54 



68 
29 
31 
48 
62 



5 
6 
3 
3 
8 



14 
8 
4 

5 

17 



Personal/Social 

VALUES, ETHICS 61 

UND SELF 72 

UNO OTHERS 79 

TEAM 45 

HEALTH , 37 



73 
84 
79 
62 
43 



12 
12 

0 
17 

6 



31 
43 
0 

31 
10 



Science 

SCI/EXP 
SCI/TECH 
QUANT TH 



31 
28 
34 



50 
49 
58 



19 
21 
24 



28 
29 
36 



Vocational 



VOC 



32 



58 



26 



38 
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APPEiiJIX TABLE 13 



PERCENTAGE OF FRESHMEN AND SENIORS REPORTING 
SUBSTANTIAL GAINS AT PUBLIC COMPREHENSIVES 

(N = 1377) 



Percentage Substantial Gain Change Change as Percentage 
GAINS Freshmen Seniors in Percent of Possible Change 



Intellectual Skills 

ANAL, LOGIC 
SYNTH. REL. 
IND LRN 
SPEC ADV ED 

General Education 

BROAD .KNOWL 

ARTS 

LIT 

WRITE 

PHILS. CULTS 

Personal/Social 

VALUES, ETHICS 
UNO SELF 
UNO OTHERS 
. TEAM 
HEALTH 

Science 

SCI/EXP 
SCI/TECH 
QUANT TH 

Vocational 

VOC 
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40 
45 
56 
32 



61 
71 
77 
74 



21 
26 

21 
42 



35 
47 
48 
62 



56 
24 
16 
41 
44 



62 
24 
23 
45 

■52 



6 
0 
7 
4 
8 



14 
0 
8 
7 

14 



49 
58 
60 
38 
29 



63 
74 
76 
51 
38 



14 

16 
16 
13 
9 



27 
38 
40 
21 
13 



26 
28 
27 



34 
34 
47 



8 
6 
20 



11 
8 
27 



29 



66 



37 



52 



APPENDIX TABLE 14 



PERCENTAGE OF FRESHMEN AND SENIORS REPORTING 
- SUBSTANTIAL GAINS AT PRIVATE CCMPREHENSIVES 

(N = 2542) 



Percentage Substantial Gain Change Change as Percentage 
GAINS Freshmen Seniors in Percent of Possible Change 



Intellectual Skills 



ANAL. LOGIC 


54 


73 


19 


HI 


SYNTH. REL. 


66 


83 


17 


50 


IND LRN 


.73 


88 


15 


56 


ortL MUV tU 


48 


Til 

74 


26 


50 


General Education 






• 




BROAD KNOWL 


65 


11 


12 


34 


ARTS 


34 


41 


7 


11 


LIT 


36 


45 


9 


14 


WRITE 


51 


59 


8 


16 


PHILS, CULTS 


54 


66 


12 


26 


Personal/Social 










VALUES, ETHICS 


64 


76 


12 


33 


UND SELF 


73 


84 , 


11 


41 


UNO OTHERS 


80 


83 


8 


15 


TEAM 


47 


59 


12 


23 


HEALTH 


42 


39 . 


- 3 




Science 










SCI/EXP 


27 


34 


7 


10 


SCI/TECH 


; 26 


34 


8 


M 


QUANT TH 


36 


47 


11 


V 


Vocational 








I 


VOC 


40 


62 


22 


37 
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APPENDIX TABLE 15 



PERCENTAGE OF FRESHMEN AND SENIORS REPORTING 
SUBSTANTIAL GAINS AT LIBERAL ARTS, TYPE I 
• (N = 1350) 





Percentage. Substantial Gain 


Chanoe 


Chdnae as Pprcentaoe 


GAINS Freshmen 


Seniors 


in Percent 

III 1 W 1 W 1 t w 


of Poss i bl G Chanae 


Intel fectual Skills, 








c 


ANAL. LOGIC 


57 


11 


20 


47 


SYNTH. REL. 


67 


84 


17 


52 


IND LRN 


68 


88 


'20 

mm Vr 


63 


prtL MUV tu 




/y 


25 


54 


General Education 










BROAD KNOWL 


79 


,83 






ARTS 


37 


49 


12 

AC* 


19 


1 TT 

LIT 




o4 


11 


19 


WRITE 


43 


64 


' 21 


37 


PHILS, CULTS 


68 


75 


7 


22 


Personal /Social 








■ 


VALUES, ETHICS 


68 


79 


11 


34 . 


UND SELF 


76 


84 


8 


33 


UNO OTHERS 


11 


79 


2 


9 


TEAM 


37 


49 


12 


19 


HEALTH 


32 


33 


1 


1 


Science 










SCI/EXP 


34 


45 


11 


17 


SCI/TECH 


34 


\44 


10 


16 


QL'.'^.NT TH 


33 


.44 




16 


Vocational 










VOC 


22 


40 


18 


23 



X 
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APPENDIX TABLE 16 



PERCENTAGE OF FRESHMEN AND SENIORS REPORTING 
SUBSTANTIAL GAINS AT LIBERAL ARTS, TYPE II 

(N = 1135) 



— 1 

Percent, Substantial Gain 
GAINS Freshmen Seniors 


Change 
in Percent 


Change as Percentage 
of Possible Change 


Intellectual Skills 










ANAL, LOGIC 


50 


65 


15 


30 


SYNTH REL. 


65 


80 


15 


43 


IND LRN 


75 


86 


11 


44 


SPEC ADV ED 


47 


71 


24 


45 


General Education 










BROAD KNOWL 


59 


64 


5 


12 


ARTS 


26 


26 


0 


0 


LIT 


35 


30 


- 5 




WRITE 


56 


56 


0 


0 


PHILS, CULTS 


58 . 


60 


o 
c 




Personal/Social 










VALUES, ETHICS 


67 


72 


5 


15 


UNO SELF 


77 


83 


6 


26 


UNO OTHERS 


78 


77 


- 1 




TEAM 


55 


63 


8 


18 


- HEALTH 


43 


40 


- 3 




Science 










SCI/EXP 


24 


34 


10 


13 


SCI/TECH 


25 


37 


12 


16 


QUANT TH 


40 


48 


8 


13 


Vocational 










VOC 


43 


72 


29 


5} 
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